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A Chip Off The Old Block, 



Good genes run in our family. 
Ampro's EPIC and Mini-ITX 
embedded motherboards have 
inherited many characteristics 
from their PC/104 and EBX 
cousins. These low cost SBCs 
offer many of the embedded 
features found in Ampros rugged 
CoreModule™, LittleBoard™ and 
ETX products. Features such as 
fanless operation, serial ports 
with RS422/485 options, dual 
Ethernet, battery-free operation, 
USB and LAN boot and a host of 
others. Plus, Ampro's industry¬ 
leading lifecycle management 
policies apply, assuring you of 
long lifecycle and consistent 
product deliveries, time after 
time, with notification of product 
changes well in advance. 

Don't let the low price fool you. If 
you're looking for an embedded 
motherboard for a cost-sensitive 
application, the best solutions 
are Ampro ReadyBoard™ and 
MightyBoard™ SBCs. With a 
lineage second to none, our 
embedded motherboards have 
Ampro written all over them. 

Quality... 

Durability... 

Dependability... 

It runs in the family. 


Ampro Embedded Motherboards 


Call today (408) 360-0200 or visit: www.ampro.com 




® 


Ampro is a registered trademark of Ampro Computers, Inc. All other trademarks are the property of their respective owners. © 2005 Ampro Computers, Inc. All rights reserved. 










Costs down. Profits up. Seems like a good idea to everyone. 

No wonder Motorola's standards-based Application-Enabling Platforms are drawing a crowd. 

By integrating AdvancedTCA®, CompactPCI ®, Carrier Grade Linux; Service Availability™ Forum and 
other industry standards into highly scalable and pre-integrated platforms. Motorola helps deliver 
significant time-to-market, cost and flexibility benefits. And by eliminating steps that don't add value, 
you can spend your time and budget on the stuff that makes your products stand out in the crowd. 
Controlling costs today, and helping to protect your long-term investment. Ready to get on board? 






MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. CompactPCI and 
AdvancedTCA are registered trademarks of the PCI Industrial Computer Manufacturers Group. Service Availability 
is a trademark of the Service Availability Forum. All other product or service names are the property of their 
respective owners. © Motorola, 2005. 







HoujM they do that? 



SBS knows Intel® Architecture. 

Pack the power of an Intel® Pentium® M processor into tight quarters. 


THESE DAYS, designers want a lot of processing power in a very 
small package. And that's exactly what SBS has delivered with 
our newest Intel Pentium M-based 3U CompactPCI® 
and PrPMC single board computers. 


These fast, compact modular building blocks 
are perfect for space-constrained applications 
like avionics, network edge, industrial and vehicle 
management systems. The PSL09 PrPMC, for example, 
runs at speeds of up to 1.4 GHz, offers up to 256 MB 
DDR SDRAM and supports Microsoft® Windows® XP, 



CR4 

3U CompactPCI Conduction Cooled 
Intel Pentium M processor 
Single Board Computer 


VxWorks®and Linux®. It is compatible with CompactPCI 2.15 
and features a high bandwidth 400 MHz bus between 
the processor and chipset. The CR4 3U board is 
even more impressive, with twice the RAM and 
a rugged, conduction cooling option. 

SBS has been designing small, powerful products like the 
PSL09 and the CR4 for a long time now, so we know how 
-l to pack big capabilities into a small space. Because of 
our experience, we know that sometimes less really 
is more. 


Technologies . 


SBS knows. Find the compact Intel Pentium M processor board you J re looking for at www.sbs.com 
or call 800.SBS.EMDEDDED 
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The major phases of reading data from a 
disk and the time associated with burst 
and sustained data transfers. • Pg. 16 
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Operate and 
survive under 
the most extreme 
conditions with 
ruggedized E-Disk® 
solid-state flash drives and 
network storage solutions. 
BiTMICRO’s cutting-edge 
storage technologies offer utmost 
reliability, optimum data security and 
unmatched performance. 


Ethernet I Fibre Channel I SCSI IIDE/ATA 
USB I FireWire I cPCI VME I SATA I iSCSI 
PCI-X I PCI Express I SAS I Infiniband 


BiTMCiW-r 

ULTIMATE STORAGE SOLUTIONS ™ 1 I 


BiTMICRO Networks, Inc. 
45550 Northport Loop E 
Fremont, CA 94538-6481 


^ www.bitmicro.com 
info@bitmicro.com 
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Your OEM product can become extinct long before its time, unless you have a long-term com¬ 
mitment from your SBC vendor. VersaLogic guarantees product availability for at least five years 
from its introduction date.^ That saves you from tossing future sales into the jaws of competi¬ 
tors because you suddenly cannot get your hands on the boards you need, when you need them. 
VersaLogic’s Extended Life-Cycle Policy is just one of the services that helped 
us earn the prestigious VDC Platinum Vendor award. From our close ties with key 
vendors to our pre-design component studies, we make it our business to extend 
your product’s life. So sink your teeth into the details at www.VersaLogic.com/ 
pv13. Start a long, long-term relationship by calling us at (800) 824-3163. 


Before Kristin Koons joined 
VersaLogic’s sales support 
team, she guided tours at 
a natural history museum. 
She really knows how 
to keep products off the 
endangered species list. 


[VDC 


Platinum Vendor award by 
Venture Development Corporation 

Platinum Vendor 2002 and 2003 


VersaLogic 

"corporation 


(800) 824-3163 
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Looking for a solution? 



We've got it! 

Electronic Solutions continues to reach new heights 
in standard off-the shelf chassis products supporting 
the Open Bus Architectures for CompactPCI, VME, 
VME64Extensions, PXI/VXI, AdvancedTCA, and 
other Switched Fabric Architectures. Our system 
level components and turnkey system solutions 
incorporate high performance interconnect 
technologies for today’s fast moving industries: 
telecommunications, networking, commercial, 
instrumentation, medical and military. 


At Electronic Solutions our focus is on the 
optimum solution in support of your unique system 
packaging requirements. We can make modified 
or custom solutions to your requirements. 


Electronic 

Solutions 

a business unit of (&JPW Ltd. 
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For more information: sales.elsol@apw.com or visit www.electronicsolutions.com 
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Flurries Around AMC 
Appear to Be Letting Up 

by Tom Williams, Editor-in-Chief 


W ell, it wouldn’t be a major new specification if it didn’t 
stir up a little contention and controversy. The Advanced 
Mezzanine Card (AMC), which has come out of PICMG 
as a companion to the ATCA, has been through a gauntlet of patent 
and change issues that may not yet be finished but which appear to 
be less threatening to its future than they had seemed earlier. 

One of the more ominous indications of trouble a couple of 
months ago was a set of change requests that apparently came 
out of Intel, which, if they had all been adopted, might have 
either split the specification or put it on hold until they were 
resolved, thus throwing the industry into a spasm of delay and 
uncertainty. Among the areas affected were the connector and 
the front panel and latching mechanism. In all, the document is 
reported to have contained some 400 major and minor changes. 
As a result it was beginning to look like the ATCA—and by ex¬ 
tension the MicroTCA—effort was going to be between a rock 
and a hard place: Either they go ahead as specified and simply 
not address certain markets, or they halt progress on the spec, 
incorporate the changes and leave behind companies that had 
already heavily invested in it, and muddy the market. At one 
point it even appeared that a separate group of companies might 
split off to implement the revised spec, thereby adding a heap of 
legal issues to the already growing mountain of technical and 
commercial issues. 

Fortunately, this scenario of multiple disasters appears to 
have been averted. While hardly anybody wants to go on the re¬ 
cord in this matter, sources tell me that proposals for changes that 
would break backward compatibility with the existing spec have 
been withdrawn. The thorny issue of the connector, for example, 
has been resolved. Originally, the spec called for a compression-fit 
connector that required it to be mounted on a gold-plated connec¬ 
tor grid on the circuit board and clamped down with screws and 
nuts. PICMG has apparently qualified at least three more connec¬ 
tor manufacturers who can now additionally supply surface mount 
and through-hole versions of the connector while maintaining 
backward compatibility. 

Get Connected with companies mentioned in this article. 

www.rtcmagazine.com/getconnected 



The connector and the mechanical issues are related to shock 
and vibration testing. PICMG reportedly does not yet have inde¬ 
pendent shock and vibe test results, but will be submitting boards 
to an independent test lab within 30 days. By that time, it should 
be known how far beyond the “NEBS-only” shock and vibration 
conditions—if at all—the spec can go. AMC was never intended 
for rugged environments such as military vehicles, but there are a 
number of ideas for using it in such areas as medical instrumen¬ 
tation and industrial control if it can meet those less-stringent 
environmental demands. Which brings us to MicroTCA. 

MicroTCA is a system in which AMC cards are mounted in 
a chassis rather than as mezzanines on a carrier card. That chas¬ 
sis can be either a 19-inch rack or an 8-inch cube. The 19-inch 
version is intended for high-availability telecom applications and 
the cube is intended for lower-cost general and industrial applica¬ 
tion. This arena has been clouded by patent claims reported to be 
coming out of Lucent. But according to sources, Lucent’s patent 
application covers only what is known as an “array of cubes,” and 
PICMG’s IP rules require anyone participating in a standards ef¬ 
fort under PICMG to disclose any patent issues that would impact 
a part of the spec. 

PICMG is reported to be concentrating first on finishing the 
high-end, HA and redundant spec and will then turn its attention 
to the lower-cost cube. Each of these will have its own technical 
issues, including shock and vibration, which will determine what 
application areas they are truly suited for. However, reports of 
impending doom appear to be premature. 

Meanwhile, the VMEbus Trade Association (VITA) has 
started up a working group for VITA 56, which will be a hot- 
swappable, front-inserting mezzanine card that might remind one 
of AMC on the surface. However, it is certain that its specification 
will be aimed at military and rugged environments from the start 
and will fall into a much more demanding and smaller—hence 
more expensive—category than AMC. There should therefore be 
little if any overlap between them when they are both out in the 
real world. It will, however, be interesting to see if MicroTCA 
might eventually encroach on areas currently dominated by 
PC/104 or possibly 3U CompactPCI. Stay tuned to RTC because 
we will be closely following all the developments, d 
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Built to go in harm’s way. 


NEW! TPPC64™ (IBM PowerPC 970FX-based SBC) 


.ilk* 


6U VME64 engine with up to two IBM® 1.8 GHz PowerPC® 970FX CPUs 
Up to 4GB of DDR400 SDRAM 
VME interface - Universe II VME64x interface 
Two 10/100/1000 Ethernet ports 
Six USB ports 

AC97 audio, two serial ports 
Support for Linux™ OS 
Ruggedized for up to 30G shock 


^ NEW! TA64™ 

(AMD Turion 64 Mobil-based SBC) 

& Low Power, AMD® Turion™ 64 Mobile CPU 
Single/2-slot VME configurations 

F f I 1 1/ \ Up to 4GB ECC DDR memory 

\ Four PMC slots, two USB ports 
T # Jf'l I Jr PS/2 keyboard and mouse port 

J I H.Jji i|V ," r Compatible with USPIIe-USB™ SBCs 

. 64-bit Solaris 10, Windows® 

^ and Linux support 

W * Ruggedized for up 

to 30G shock 


USP llli™ VMEbus 
Computer 


Two UltraSPARC® llli 
1.2 GHz CPUs 
4GB 266DDR SDRAM 
per CPU, 8GB total 
Optional TGA3D+™ graphics 
Gigabit Ethernet port 
One 64-bit/66-MHz PMC slot i 
Up to three PMC ' 
expansion slots 
Dual FC-AL ports 
Additional VME- 
backplane access 
Up to 30G shock 
Solaris™ OS 


•IF 


i 


M 


Our ultra-rugged, mission-critical SBCs 
now support more technologies than ever. 

Extreme environments that need mission-critical reliability need 
Themis’ single-board computers. 

Our high performance, high availability computers have proven 
themselves in the harshest, most demanding applications and operating 
environments. Where other systems fail, our systems survive - 
in extreme temperatures and up to 30G shock. 

And now Themis’ family includes AMD and IBM PowerPC processors 
in addition to our well-known UltraSPARC VME and Compact PCI 
single board computers and graphic accelerators. So we can support 
applications in Solaris, Windows, Linux, UNIX®, and more. 

All are designed for maximum flexibility and low Total Cost of Ownership. 
Because the budgeting process can be a pretty hostile environment, too. 





So when your mission takes you in harm’s way, you can rely on Themis. 

www.themis.com (510)252-0870 


THEMIS 


Transformational. 


D 2005. Themis Computer, Themis, Themis logo, TA64, TGA3D+, USPIIe-USB and USPIIIi are trademarks or registered trademarks of Themis Computer. All other trademarks are property of their respective owners 
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Xilinx Targets $2 Billion High-Performance DSP Market 
with FPGA Solution Roadmaps 

Xilinx has unveiled a strategy and set of product roadmaps to address what it estimates as a 
$2 billion high-performance DSP market based on a series of application-optimized XtremeDSP 
solutions. Xilinx is initially targeting high-growth digital communications; multimedia, video and 
imaging (MVI); as well as defense systems segments, which combined represent more than 80 
percent of this so-called high-performance DSP market. 

The Xilinx roadmaps encompass core technologies—such as orthogonal frequency divi¬ 
sion multiplexing (OFDM) and code division multiple access (CDMA) for the digital communica¬ 
tions and defense segments and video codecs for MVI applications—needed to deliver pre-pack¬ 
aged, easy-to-use development solutions tailored for each of the target markets. 

The XtremeDSP roadmaps target Xilinx solutions more specifically at high-performance, 
application-specific system development. For digital communications, they will provide advanced 
algorithms for OFDM and W-CDMA standards that are evolving to support multiple input multiple 
output (MIMO) antenna schemes and increasingly sophisticated baseband and radio subsystems 
sporting interference cancellation and multi-user detection schemes. 

The MVI roadmap addresses needs in a wide range of segments, including consumer, 
automotive, broadcast, enterprise IT, surveillance and medical. Key components in this area are 
development platforms for accelerating video/image processing system design, advanced video 
codecs for standard and high-definition applications (e.g., FI.264 encoding) and a comprehensive 
video IP library. Xilinx has also announced separately today the release of the first in a series of 
MVI-related deliverables and solutions. 

The defense systems roadmap targets market segments such as military communica¬ 
tions, intelligence and sensors. In particular, Xilinx will focus on applications including software- 
defined radio, wideband analysis, direction finding, jamming, radar and sonar. Key solutions span 
the spectrum from enabling technologies, such as partial reconfiguration to custom waveforms 
and components, as well as pre-configured JTRS development kits, complete with an entire 
Software Communications Architecture (SCA) operating environment and tool suite for waveform 
development. 


VITA 56 Mezzanine 
Standard Working Group 
Established 

The VMEbus Industry 
Trade Association (VITA) 
has initiated a new mezzanine 
standard working group referred 
to as “VITA 56.” VITA 56 is 
a predominately PCI Express- 
based mezzanine board similar in 
dimensions to a PCI Mezzanine 
Card (PMC), allowing it to be 
used on VME and CompactPCI 
boards. These boards will be 
fully de-pluggable/hot-swappable 
without removing the carrier, 
will offer an advanced level of 
manageability and will include 


support for many high-speed 
serial fabrics. 

Among the goals is to create 
a mezzanine standard that can 
be used in military, COTS and 
rugged industrial environments 
for which the current AMC 
standard is not intended. 
According to VIITA executive 
director Ray Alderman, “VITA is 
the crucible for technologies for 
critical applications where lives 
are at stake. We don’t give a damn 
what telecom wants.” 

Envisioned VITA 56-based 
boards will cover a wide range 
in terms of their functionality. 
VITA 56 enables a modular 


building block design for both 
industry standard and proprietary 
Host Boards. This specification is 
intended to enable larger markets 
with more unique functions and 
creates economies of scale that 
lower prices. 

Tsunami Alarm System 
over Mobile Phones 
Realized 

In the future, travelers will 
be warned of catastrophes like 
the tsunami that occurred a year 
ago in Asia, with the worldwide 
unique Tsunami Alarm System 
for everyone within reach of a 
GSM mobile phone network. 
The German professors Eduard 
Heindl and Wolfram Reiners 
instigated the project and it is 


being executed under 3MFuture 
Ltd., Heindl Internet AG and 
PhiBlue Mobile Ltd., under the 
auspices and presidency of the 
instigators Heindl and Reimers. 

Measuring stations all 
around the world operate day and 
night to be able to warn quickly 
and reliably of tsunamis: Seismic 
sensors measure the earth 
tremors. Pressure and velocity 
sensors in the ocean detect fast 
changes of water bodies in the 
sea. Advance warning systems 
check first alarm signals. Users 
can subscribe to the Tsunami 
Alarm System by simply entering 
the number of their mobile phone 
on the Web site, thus quickly 
enabling the alarm system on their 
phone. Nothing has to be installed 
or downloaded. The idea of a 
Tsunami Alarm System arose in 
December 2004 in the aftermath 
of the devastating destruction of 
the tsunami catastrophe in the 
Indian Ocean. 

Green Hills Software 
and Navteq Team 
for Telematics and 
Infotainment Development 

Green Hills Software has 
announced that it has established 
an agreement with Navteq, a 
provider of digital map data 
for vehicle navigation and 
location-based solutions, to offer 
integrated solutions to developers 
of telematics and infotainment 
systems requiring navigation. As 
part of the agreement, Navteq 
map data and other technologies 
have been integrated with Green 
Hills Software’s royalty-free 
Integrity and velOSity real-time 
operating systems. The combined 
solution from Navteq and Green 
Hills Software has already been 
selected by several tier 1 suppliers 



•SOFTWARE, INC* 
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Event 

Calendar 

11/30-12/01/05 

GVExpo2005 
Washington, DC 
www.gvexpo.com 

12/01/05 

Real-Time & Embedded 
Computing Conference 
Portland, OR 
www.rtecc.com/portland 

12/05-08/05 

TechNet Asia-Pacific 
Int’l Conf & Expo 
Honolulu, HI 
www.afcea.org 

12/06-08/05 

AdvancedTCA Summit 
San Jose, CA 
www.atcaseminar.com 

12/06/05 

Real-Time & Embedded 
Computing Conference 
Seattle, WA 
www.rtecc.com/seattle 

12/08/05 

Real-Time & Embedded 
Computing Conference 
Vancouver, BC 
www.rtecc.com/vancouver 

01/16-17/06 

Bus&Board Conference 
Long Beach, CA 
www.busandboard.com 

01/26/06 

Real-Time & Embedded 
Computing Conference 
San Jose, CA 
www.rtecc.com/sanjose 


If your company produces any 
type of industry event, you can 
get your event listed by contacting 
sallyb@rtcgroup.com. 

This is a FREE industry-wide listing. 


to the automotive industry, and 
deployment is targeted for certain 
vehicles for model year 2006. 

Integrity is an RTOS 
designed for applications that 
require scalability, small size, 
certifiable reliability and real¬ 
time responsiveness. Built on the 
velOSity microkernel, Integrity is 
suited for cost-sensitive, resource- 
constrained applications with 
demanding performance and 
reliability requirements, such as 
consumer and automotive devices. 

Navteqprovides comprehensive 
digital map information for 
automotive navigation systems, 
mobile navigation devices, Internet- 
based mapping applications, 
government and business solutions. 
Navteq creates the digital maps and 
map content that power navigation 
and location-based services solutions 
around the world. 

PICMG Adds Serial RapidIO 
to AdvancedTCA 

PICMG has announced the 
release of its latest specification 
for high-speed interconnects 
over AdvancedTCA (ATCA) 
backplanes—PICMG 3.5 RapidIO 
for ATCA. PICMG 3.5 was 
recently approved by the PICMG 
membership and is now available. 

The PICMG 3.0 specification 
defines the detailed characteristics 
of AdvancedTCA cards, chassis 
and backplanes as well as the 
protocol for the base interconnect 
between cards (Ethernet), but 
leaves the protocols on the extended 
fabric to other specifications. An 
add-in card must notify the system 
manager which fabric it supports 
before interconnects are enabled 
onto the backplane. Serial RapidIO 
is the latest fabric to be mapped 
onto AdvancedTCA. 

The RapidIO interconnect 
architecture is an established, 
open standard available for 
review and downloading from 
the RapidIO Trade Association’s 
Web site, http://www.RapidIO.org. 
Also available at the Web site is 
information on system-enablement 
tools including RapidIO vendor 
product lists, synthesizable Verilog 
cores, analog physical layer cores, 


logic and protocol analyzers, 
operating system support, bus 
functional models and hardware 
interoperability platforms. 

Copies of the complete 
PICMG 3.5 specification are 
available to PICMG members and 
can be purchased by non-members 
from PICMG. More information 
on new PICMG developments is 
available at http://www.picmg. 
org/picmgnewinitiatives.stm. 

Mercury Joins Xilinx to 
Improve Ease of Use for 
FPGA-Based Products 

Mercury Computer Systems 
has announced that it has 
joined Xilinx Corporation in 
membership to the Open Core 
Protocol International Partnership 
(OCP-IP), to help evolve the 
standard in support of FPGA- 
based system designs. The OCP- 
IP is dedicated to promulgating a 
common standard for intellectual 
property (IP) core interfaces, or 
sockets, which facilitate “plug 
and play” System-on-Chip (SoC) 
design. Formed in 2001, OCP-IP 
is a non-profit corporation that 
facilitates IP core reusability 
and reduces design time, risk 
and manufacturing costs for SoC 
designs. 

Historically, customers who 
wish to interconnect IP cores have 
been burdened with the creation 
and maintenance of proprietary 
hardware APIs (application 
programming interfaces) to 
support the connection. Adherence 
to the standard interfaces defined 
by OCP is expected to enable 
Mercury customers to seamlessly 
integrate Xilinx and other third- 
party IP cores into their FPGA 
designs, with particular emphasis 
on reuse and verification. OCP-IP 
interfaces add value by providing 
a platform that brings together a 
wide range of applications and 
architectures. Mercury and Xilinx 
intend to extend this proven 
technology, which has already 
been used extensively in the 
ASIC/SoC space, to their FPGA 
customers. For more information 
on OCP-IP, visit www.ocpip.org. 


Program Launched to 
Certify Printers for RFID 
Antenna Production 

Parelec Inc., a manufacturer 
of additive inks for electronic 
printed circuits, has developed 
a program for customers in the 
RFID value chain. This program 
permits customers to acquire 
RFID antennas from Parelec’s 
proven Certified Printer Partners 
for inlays, labels, packaging and 
other applications with a rapid 
turnaround and at low cost. 

Under the Parelec program, 
any eligible customer who is 
looking to implement RFID 
projects will have access to 
Parelec’s trained and certified 
printers who manufacture RFID 
labels, companies experienced 
in tag assembly operations 
and integration. The program 
promotes cost efficiencies and 
partnership opportunities that 
will simplify and speed up RFID 
project implementation. 

Parelec applies nano¬ 
technology and advanced 
materials systems to develop 
and market conductive inks 
and related materials for the 
manufacture of electronic circuits. 
The company’s core product 
lines, Parmod inks and pastes and 
Modflex films, are used to provide 
low-cost RFID antenna, specialty 
and flexible circuits, intelligent 
packaging, heaters/defrosters for 
plastic windows, and are used in 
semiconductor packaging for a 
wide variety of electronics and 
consumer products. 
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High-Speed Storage Interface 


Parallel and Serial 

Storage Interfaces 

In embedded systems storage, a transition is taking place from parallel 
storage interfaces to high-speed serial interfaces. However, serial interfaces 
va?y widely in terms of connectivity, bandwidth and performance scalability. 


by Wieslaw Wojtczak 
Adtron 


T oday’s embedded applications re¬ 
quire greater performance in stor¬ 
age systems, putting pressure on the 
storage industry to improve data band¬ 
width for more effective usage of operat¬ 
ing systems and embedded applications. 
The evolution from parallel interfaces, 
such as IDE and SCSI, to high-speed se¬ 
rial interfaces is currently taking place. 
The same demands for performance, sig¬ 
nal integrity, power management, seal- 
ability and cost reduction that drive the 
evolution of serial interfaces are also af¬ 
fecting storage systems. 

Attempting to achieve higher per¬ 
formance from parallel buses by merely 


padding more bits or increasing transfer 
rates is not practical, considering the con¬ 
current system requirements for smaller 
backplanes, fewer and lower power bus 
drivers, reduced connector and cabling 
costs, and reduced electromagnetic emis¬ 
sions, along with product development 
time-to-market pressures. 

When compared to their parallel 
counterparts, high-speed serial interfaces 
reduce connector costs, consume less 
power, simplify cabling and associated 
EMC issues and actually deliver improved 
system performance (Figure 1). Another 
significant advantage is their ability to 
operate over long distances between con¬ 


nected devices. In an extreme example, an 
Ultra 320 SCSI cable has a limitation of 
three meters compared to a 1 Gbit iSCSI 
interface that can target transfers over 
thousands of kilometers using Internet 
protocols. The contrasts between paral¬ 
lel interface bus architectures and serial 
interface bus architectures illustrate the 
improved connectivity and performance 
enhancements of serial interfaces. 

The Parallel Interface Life 
Cycle 

The most common hard disk inter¬ 
faces today are SCSI and Parallel ATA 
(PATA)/IDE. SCSI is the most commonly 


Bus bit rate 
(burst rate) 

SCSI 

(P)ATA/IDE 

SAS 

SATA 

USB 

FC 

iSCSI 

2.5 Gbits/s 
(320 Mbytes/s) 

800 Mbits/s 
(100 Mbytes/s) 

3.0 Gbits/s 

1.5 Gbits/s 

480 Mbits/s 

2 Gbits/s 

1 Gbit/s 

Next version 

SAS 

SATA 

6.0 Gbits/s 

3.0 Gbits/s 

-- 

4 Gbits/s 

10 Gbits/s 

Cable length 

3 meters 

1/2 meter 

1 meter 

1 meter 

1 meter 

kilometers 

kilometers 

Disk hot swap 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

Form-factor 

3.572.5” 

3.572.5” 

3.5”/2.5” 

3.5”/2.5” 

External 

3.5” 

External 

Flash disk 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

No 

Target market 

Enterprise 

PC 

Enterprise 

PC 

Consumer 

Enterprise 

Enterprise 

Relative cost 

$$$ 

$$ 

$$$ 

$$ 

$ 

$$$ 

$$$$ 

Parallel and serial interfaces compared. 
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Figure 2 


In a simplified computer system, the host interface controls access and 
data transfer to the storage media. 


used interface for high-performance disks, 
tapes and storage systems. 

The SCSI Ultra 320 version sup¬ 
ports maximum burst transfer rates of 320 
Mbytes/s and a 16-bit bus. The SCSI con¬ 
troller that connects the storage media and 
the CPU manages a complex parallel pro¬ 
tocol to deliver storage deployment seal- 
ability by efficiently supporting up to 15 
devices and one controller. The SCSI bus 
protocol allows computer systems to op¬ 
timize parallel transfers through the use 
of disconnect and command queuing fea¬ 
tures not used in IDE buses. Thus, SCSI 
was generally adopted for performance- 
focused applications, such as enterprise 
servers with high-availability RAID re¬ 
quiring 24-7 enterprise-class operation. 

Because of the interface’s complex¬ 
ity, SCSI-based devices are relatively 
expensive compared to IDE devices and 
have not found wide acceptance outside 
the enterprise. 

The PATA interface, commonly 
called ATA, is also known as IDE. Cre¬ 
ated for the IBM XT in 1982, it was stan¬ 
dardized by ANSI. At the time, this de¬ 
sign was the simplest parallel I/O, direct 
attached solution to deliver low cost stor¬ 
age for the emerging PC market. This ba¬ 
sic design remained in all x86-based PCs 
until recently, when Serial ATA (SATA) 
replaced it as the next standard. 

Today’s PATA UDMA Mode 5 ver¬ 
sion supports burst data transfer rates up 
to 100 Mbytes/s across a 16-bit bus. PATA 
is limited by lack of scalability; only two 
PATA devices can be supported in master/ 
slave configuration over a single channel. 
PATA disks connect via a low cost 40- or 
80-pin ribbon cable to a PATA control¬ 
ler. Compared to the SCSI controller, the 
PATA controller is simple and generally 
found integrated into standard x86 chip 
sets. Thus, the ATA interface effectively 
“comes for free.” 

PATA disk drive production quanti¬ 
ties eclipse SCSI quantities. PATA is by 
far the most commonly used disk inter¬ 
face in industrial embedded computers. 
Only recently have PATA disks moved 
into non-critical, low-cost business serv¬ 
ers. Without the enterprise and perfor¬ 


mance advantages found with the SCSI 
protocol, PATA will probably remain 
in low-end small office and home office 
(SOHO) storage systems. 

Both PATA and SCSI use a parallel 
bus that was sufficient for early deploy¬ 
ments with limited transfer rates and 
relatively easy implementations. Today’s 
high-speed data transfer requirements 
push PATA capabilities to the limit, while 
making SCSI implementations very com¬ 
plex and expensive. 

It is important to note another evo¬ 
lution in the storage industry, from Direct 
Attached Storage (DAS) to Network At¬ 
tached Storage (NAS) architectures. Net¬ 
work Attached Storage commonly refers 
to storage connected to a network or fab¬ 
ric, without defining a specific protocol. 
High-speed serial interface standards with 
network connectivity include Fibre Chan¬ 
nel (FC), Fibre Channel Arbitrated Loop 
(FC/AL), InfiniBand, RapidIO and Eth- 
ernet/IP-based solutions including Inter¬ 
net SCSI (iSCSI), Internet Fibre Channel 
Protocol (iFCP) and Fibre Channel over IP 
(FCIP). Telecommunications and data-crit- 
ical storage centers are driving these NAS 
storage systems. Recently, designers of 
commercial aircraft and military systems 
have found these network storage systems 
advantageous in their applications. 

Serial Interface Development 

Serial interfaces include SATA, Se¬ 
rial Attached SCSI (SAS), FC and USB. 


Serial ATA incorporates the same com¬ 
mand and data protocols as PATA. From 
the standpoint of function, feature and 
operating system, SATA is a drop-in re¬ 
placement for PATA with the aim of over¬ 
coming PATA’s connectivity and future 
performance limitations. 

Serial ATA uses a point-to-point ded¬ 
icated bus and eliminates the master/slave 
shared bus. It employs high-speed serial 
cable in lieu of the low reliability paral¬ 
lel bus ribbon cable. Instead of legacy 5 V 
signals to increase signal integrity and 
provide compatibility with 3.3V-based 
systems, PATA uses Low Voltage Differ¬ 
ential Signaling (LVDS). 

Moreover, SATA enables higher 
transfer rates. The first SATA release sup¬ 
ports 1.5 Gbits/s with a maximum data 
rate of approximately 150 Mbytes/s. The 
second release, planned for this year, 
will double the interface bandwidth to 3 
Gbits/s. The SATA roadmap calls for the 
third generation to support up to 6 Gbits/ 
s. With additional functionalities such as 
hot-swap capability, SATA is well posi¬ 
tioned not only to replace PATA but also 
to move into entry-level RAID and high- 
performance applications. 

Serial Attached SCSI is a succes¬ 
sor of parallel SCSI. This new interface 
maintains the reliability, performance and 
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Sustained 


Command 


Data 


Status 



Burst 


Figure 3 


The major phases of reading data from a disk and the time associated 
with burst and sustained data transfers. 


robust command set of SCSI, provides the 
benefit of higher bus speeds offered by a 
serial bus and includes SATA functions. It 
uses the same physical layer interface as 
SATA drives, including the connector, as 
well as LVDS. Utilizing a dual-function 
controller and storage device drivers, sys¬ 
tem integrators can choose either SATA 
or SAS for their storage systems. 

The first SAS devices are designed to 
support a wire rate of up to 3 Gbits/s, with 
a roadmap to achieve 6 Gbits/s within the 
next two to three years. For scalability, 
SAS allows expanders and protocol tun¬ 
neling. The dual-port feature enhances 
the use of SAS in high-availability RAID 
systems. 

The Fibre Channel interface marries 
storage and network architectures. It has 
a layered architecture following the OSI 
model, supports the full SCSI command 
set and allows for several network con¬ 
figurations including point-to-point and 
arbitrated loop. It supports serial bi-di¬ 


rectional and point-to-point data channel 
for rates of up to 2 Gbits/s with the first 
4 Gbit/s interfaces being built today and 
a roadmap to reach 10 Gbits/s. Although 
FC is not a routable protocol, it has many 
features for mapping and managing work¬ 
ing systems. Fibre Channel allows the 
scalability and connectivity sophistication 
demanded in high-availability environ¬ 
ments. Compared to SCSI and IDE, these 
features command very high prices. 

The USB serial interface, omnipres¬ 
ent in the consumer market, can deliver 
up to 480 Mbits/s of throughput. However, 
when used in storage applications, its ef¬ 
fective rate drops to around 20 Mbytes/s 
due to the USB storage command over¬ 
head. This limitation prevents USB from 
being a serious contender against SATA 
in SOHO and embedded storage systems. 

These new high-speed interfaces 
highlight the existing bottlenecks in to¬ 
day’s storage systems, such as latency and 
the resulting low sustained transfer rate 


from a rotational disk drive, as well as the 
ability of a computer system to rapidly 
move data to and from storage media and 
system memory. Cabling, air flow, power 
management and package efficiencies are 
also significant factors in the transition 
from parallel to serial. 

Bus Performance 
Considerations 

The major steps involved when read¬ 
ing data from a disk and the time associ¬ 
ated with burst and sustained data trans¬ 
fers are simplified in the examples shown 
in Figures 2 and 3. 

First, to read data from the storage 
media, the CPU sends control commands 
to the host interface requesting the data 
transfer. This command phase includes 
the CPU, host controller and disk com¬ 
mand overhead, and is shown as com¬ 
mand phase latency in Figure 3. Upon 
completion of this phase, the data is sub¬ 
sequently transferred from the media to 
the high-speed memory. 

The data transfer can be accom¬ 
plished through the CPU or the memory 
access DMA into the system memory. 
This data phase is depicted as data phase 
latency in Figure 3. 

In the last phase, the CPU waits for 
the acknowledgement from the host con¬ 
troller that the transfer is completed be¬ 
fore the newly received data in memory 
can be considered valid. This phase is 
shown as status latency in Figure 3. Upon 
its completion, the CPU can process the 
information. 

The write from the high-speed mem¬ 
ory to the media follows a similar process. 
The actual duration of the data phase de¬ 
fines the maximum data transfer rate, or, in 
a perfect system, the burst rate. The overall 
time to move the data from the storage me¬ 
dia to the high-speed memory determines 
the sustained data transfer rate. 

Should more than one storage me¬ 
dia device be used, the CPU must send 
the control commands to each device. 
The commands are sent sequentially. 
However, in SCSI/SAS/FC drives, the 
commands can be overlapped while each 
commanded disk performs the task of re¬ 
trieving its piece of the data. Data trans¬ 
fers are then conducted sequentially, add¬ 
ing latency, and hence reducing the sus¬ 
tained data transfer rate. 
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The computer system design and the 
performance of its elements clearly influ¬ 
ence the sustained rate of the data transfer 
to the storage media. The performance of 
the storage media used also has an impact 
on overall storage system design and its 
performance. 

Performance Scalability of 
Storage Media 

The inherent limitations of rotational 
disks are due to mechanical system laten¬ 
cies characterized by seek times and rota¬ 
tional latency. In addition, the bit density 
and rotational speed of the platter directly 
influence the bit rate from the media. 

The recent migration from linear to 
vertical recording methods provides a net 
bit density increase and results in media- 
to-head performance improvements. This 
has improved the maximum performance 
ceiling of rotational disks. However, since 
this is a single bit stream with the overhead 
burden of magnetic formats, the absolute 
transfer rate of the disk drive is severely 
limited. Higher bit densities and platter 
rotational speeds improve performance, 
but they are not easily scaled. 


The recent proliferation of flash 
memory solid-state disks may offer some 
advantages from the standpoint of seal- 
ability. 

Some legacy flash disk architectures 
make use of the front-end cache to accel¬ 
erate data writes with limited assistance 
for reading (read data must already be 
cache resident). However, cache-based ar¬ 
chitecture does not scale easily. Increasing 
the cache requires a more robust system to 
reduce the data loss when power is lost. 
Caching has advantages in applications 
that transfer data in small blocks. 

One significant contributor to per¬ 
formance can be found in flash disks 
that incorporate a parallel array of flash 
memory. This parallel process, already 
well understood in disk array subsystems, 
allows overhead to be managed in parallel 
and to significantly scale performance. 

A flash disk architecture that ad¬ 
dresses performance scalability makes use 
of a media controller capable of managing 
multiple flash arrays in parallel (Figure 
4). This architecture scales very well and 
can match the new serial interface speeds. 
Parallel processing of flash arrays, such as 


Adtron’s ArrayPro technology, delivers a 
150 Mbyte/s sustained data transfer rate 
without using a cache and can be scaled 
further as required. The parallel array 
structure of the flash disk may suffer some 
performance penalty when transferring 
many small blocks. Large blocks of data 
transfer much more quickly in a cache¬ 
less architecture. 

The performance enhancements and 
connectivity possibilities for high-speed 
serial buses significantly affect the effi¬ 
ciency of the overall system. High-speed 
serial interfaces must undergo comple¬ 
mentary changes in system architecture 
in order to truly deliver greater perfor¬ 
mance. d 
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High-Speed Storage Interface 


SAS and Embedded 
RAID-on-Motherboard 
Deliver Data Security 


The increased ease of implementing RAID technology has led to the 
proliferation of RAID data protection. By combining embedded RA1D- 
on-motherboard solutions with Serial Attached SCSI, system developers 
are implementing robust hardware RAID solutions while optimizing server 
motherboard investments. 


by Paul Griffith 
Broadcom 


T he demand for denser, less-expen¬ 
sive, higher performing storage so¬ 
lutions has led to the deployment 
of embedded RAID-on-motherboard 
(ROMB) technology in more and more 
systems. Solutions based on this tech¬ 
nology, using integrated RAID-on-chip 
(RoC) devices, offer higher performance, 
availability and reliability at a lower cost 
than add-in RAID solutions, such as host 
bus adapter (HBA) RAID and modular 
ROMB/Zero-Channel-RAID (ZCR). This 
is because they do not duplicate compo¬ 
nents that already exist on the mother¬ 
board and because they are, by design, 
made to interoperate with other mother¬ 
board components. 

As the successor to parallel SCSI, 
Serial Attached SCSI (SAS) has emerged 
to deliver higher levels of reliability for 
mission-critical transactional applications 
that require 24/7 online access and no 
data loss. The highly scalable and flexible 
SAS architecture allows RAID topologies 
to support multi-node clustering for high- 
availability failover and load balancing, 


and is also compatible with lower-cost Se¬ 
rial ATA (SATA) drives. This significant 
feature enables system builders the flex¬ 
ibility of integrating either SAS or SATA 
devices while dramatically reducing the 
cost to support two separate interfaces. 

By combining the flexibility and 
availability features inherent in SAS 
with the reliability, performance and cost 
benefits of ROMB, these ROMB storage 
solutions can be effectively included in 
mainstream server, storage and applica¬ 
tion markets, increasing the choices and 
design considerations available to mother¬ 
board developers and system builders. 

Main Components of a Typical 
RAID Controller Design 

The major components that comprise 
a RAID controller subsystem include the 
I/O processor (IOP), a dedicated XOR 
engine and an I/O controller (IOC). 

RAID software is executed within 
the IOP to manage such tasks as disk vir¬ 
tualization, cache processing and logical 
volume configuration. Because of this 


dedicated RAID functionality, the archi¬ 
tectural design of the IOP typically pro¬ 
vides a bigger impact on the performance 
of the RAID code—which is designed 
specifically for the IOP—than would the 
incorporation of a higher clock frequency. 
The IOP frees the host CPU from interim 
interrupts generated by I/O requests to 
member disks that are configured in stor¬ 
age arrays. It only interrupts the host CPU 
once during an I/O request, regardless of 
the total number of disks that reside within 
the storage array. 

Typically, the IOP is the only com¬ 
ponent in the RAID subsystem known to 
the host; all other RAID components are 
kept hidden from it. When the host identi¬ 
fies a RAID subsystem, it must recognize 
and communicate with the IOP alone. 
This level of abstraction effectively hides 
RAID traffic above the IOP by not pre¬ 
senting it to the system or host. 
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offers the flexibility of upgrades to future technologies, but can introduce 
other issues including incompatibilities and performance sharing. 


The second major component in a 
RAID subsystem is the dedicated XOR 
engine. By performing a simple Boolean 
XOR operation, a single disk can protect 
data on any number of associated disks. 
For example, under RAID5, the XOR en¬ 
gine creates parity data that is the result 
of an XOR operation on all other data ele¬ 
ments within the same stripe. The XOR 
function can be implemented in dedicated 
hardware such as an XOR ASIC engine, 
or as part of an IOP that may have inte¬ 
grated XOR functionality. This dedicated 
function within the hardware greatly 


increases the throughput of data requiring 
this operation. 

The RAID subsystem’s third major 
component, the I/O controller, communi¬ 
cates directly with individual disks. When 
the IOP makes a request for data, it directs 
the IOC to retrieve the data from a physical 
disk and return the data to either the IOP or 
the host, depending on direction from the 
application. The IOP never communicates 
directly with the disks. Instead, it leaves 
that task to the IOC, which can support one 
of several disk interface technologies such 
as Fibre Channel, SATA or SAS. 



modular RAID-on-motherboard (ROMB) designs reduce duplicate 
components. 


Choosing and Implementing 
the Proper RAID Design 

As intelligent RAID technology con¬ 
tinues to grow in popularity, several dif¬ 
ferent types of RAID implementations 
must be considered to achieve the great¬ 
est balance of performance, flexibility 
and reliability, all at an affordable cost 
to meet the specified design goal. At first 
glance, implementing RAID directly on a 
server’s motherboard may appear to have 
drawbacks. However, in comparison with 
other typical RAID implementations, em¬ 
bedded ROMB designs are clearly at the 
forefront. The three most common RAID 
implementations are HBA RAID, modu¬ 
lar ROMB and Zero Channel RAID. 

Host bus adapter expansion cards that 
plug into servers through a PCI-X or PCI 
Express connector are the most common 
method of implementing RAID in a server 
system. They also tend to be more flexible 
than other approaches (Figure 1). Using 
this method, upgrading a RAID controller 
is as simple as replacing the RAID HBA 
card. Although RAID HBA cards are 
available from a variety of vendors, most 
of them are not interoperable across ven¬ 
dors. When these cards are incompatible, 
administrators are forced to back up all of 
their data, replace the HBA, re-create the 
previous RAID volumes and then restore 
all of the data and applications. 

Performance can also be affected 
when inserting an HBA card into a PCI 
slot that is shared with other components 
within the system. Although most servers 
have multiple PCI buses, it may be diffi¬ 
cult to determine which components are 
sharing buses with other slots or devices. 
If two devices share the same bus this can 
easily create a bottleneck in the system, 
slowing performance. Host bus adapter 
add-in cards are typically the most ex¬ 
pensive option when implementing RAID 
into a server platform. 

Modular ROMB and Zero Channel 
RAID are essentially the same technology 
implementation that addresses some of the 
drawbacks of using an HBA card. With 
this technology, the IOC is not located 
on the PCI add-in card, but embedded di¬ 
rectly on the motherboard. As a result, the 
modular RAID HBA is still required to 
complete the RAID subsystem. However, 
since the IOC is already available on the 
motherboard, no IOC is necessary on the 
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with stability and reliability in the compact design necessary for today’s 
smaller compute densities. 


HBA (Figure 2). Special additional logic 
is still required on the motherboard to 
allow for proper interaction between the 
IOP and IOC when the modular HBA is 
present. 

Some previous ZCR systems have 
failed to completely hide the IOC from 
the host, requiring the end user to disable 
certain BIOS settings and avoid loading 
certain device drivers. Otherwise, the IOC 
could be the source of conflicts between 
the ZCR card and a host application. 

Modular ROMB was created to pro¬ 
vide a less expensive RAID alternative 
to an HBA card, since it takes full ad¬ 
vantage of the IOC that already exists on 
the motherboard, eliminating the need for 
the IOC on the HBA. However, modular 
ROMB solutions can suffer from other 
drawbacks. These include compatibility 
problems, loss of use of the IOC on the 
motherboard, loss of flexibility and lower 
performance. 

When a modular ROMB card is in¬ 
serted into the system, the motherboard’s 
IOC can no longer be used for simple drive 
connectivity, which is the likely reason it 
was there in the first place. Therefore, if 
that IOC is required for other purposes, it 
may not be possible to upgrade the system 
to modular ROMB. 

Modular ROMB cards cannot be 
added to any slot in the system: they re¬ 
quire slots designed for the particular ven¬ 
dor’s card. These special slots allow the 
cards to correctly interact with the system 
so they can use the motherboard’s IOC. 

Finally, modular ROMB performance 
can suffer because of the IOC placement 
on the motherboard. If the data flow must 
cross the PCI bus when moving between 
the IOC and the modular ROMB, then 
performance will be affected. If the IOC 
could be placed on the motherboard where 
the path between the modular ROMB and 
the IOC does not cross this PCI bridge, 
then performance would not be affected. 
However, this is difficult to do and there¬ 
fore uncommon. The existing configura¬ 
tion presents a data flow pattern that can 
double the amount of traffic on the PCI 
bus compared to a RAID HBA card or 
an embedded RAID chip on the mother¬ 
board. Although modular ROMB systems 
are designed with the knowledge of this 
limitation, the data flow can still create a 
system bottleneck. 


Embedded RAID-on- 
Motherboard Design 

Embedded RAID-on-motherboard 
design places every piece of the RAID 
subsystem directly on the motherboard, 
with most components available in a single 
ASIC. This approach provides the high¬ 
est performance and stability (Figure 3). 
The primary disadvantage of embedded 
ROMB is the lack of hardware upgrade- 
ability, although features and performance 
can generally be upgraded in firmware. 
The flexibility of HBA-based RAID has 
long been challenged by the simple price 
advantage of embedded ROMB. By em¬ 
bedding the HBA directly on the moth¬ 
erboard, the overall solution price can be 
substantially lower than the cost of buying 
components separately. 

System designers can optimize the 
motherboard layout for the RAID subsys¬ 
tem since the components are fixed with 
regard to each other, the PCI buses and 
other devices. As a result, designers can 
create a ROMB system in which other 
PCI devices cannot interfere with RAID 
traffic, optimizing throughput and elimi¬ 
nating performance bottlenecks. 

Because the RAID subsystem is 
designed directly into the motherboard, 
it provides greater stability. Typically, 
system designers rely on industry stan¬ 
dard specifications and validation to 
ensure interoperability. In addition to 
the benefits of open standards, ROMB 
subsystems can also avoid interoper¬ 
ability problems by fixing the location 
and selection of such major compo¬ 
nents as the IOP and IOC. Doing so re¬ 
duces interoperability issues and helps 


ensure the maximum amount of valida¬ 
tion time, since the entire configuration 
is validated as part of the motherboard. 
Furthermore, because these components 
are embedded on the motherboard, they 
typically receive better airflow and cool¬ 
ing, which also contribute to improved 
system reliability. 

Maximizing RAID Availability 
with SAS 

Serial Attached SCSI, the successor 
technology to the parallel SCSI interface, 
leverages proven SCSI functionality and 
promises to build upon the existing ca¬ 
pabilities of the enterprise storage con¬ 
nection. It offers many features not found 
in today’s mainstream storage solutions. 
These include drive addressability of up 
to 16,384 devices and reliable point-to- 
point serial connections at speeds of up to 
3 Gbits/s. 

Serial Attached SCSI has emerged 
to deliver higher levels of reliability than 
previous generations of SCSI for mis¬ 
sion-critical transactional applications 
that must be online around the clock with 
no data loss. When combining SAS with 
RAID, multiple disk drives can be used, 
together with fault-tolerant designs, to 
create highly reliable, high-performance 
subsystems. 

A key advantage of SAS is that its 
backplane design and protocol interface 
allow both SAS and SATA drives to be 
used in the same system. Although each 
kind of drive is typically used for differ¬ 
ent applications, most enterprise users 
have needs for both technologies. The 
ability to mix and match these drives is a 
powerful benefit for designers and users. 
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Serial ATA drives are designed pri¬ 
marily for low-cost bulk storage where 
transaction rates are low and data avail¬ 
ability is not mission-critical. As a re¬ 
sult, SATA drives feature lower spindle 
speeds, typically 7,200 rpm, and a lower 
mean-time-between-failures (MTBF) 
than SAS drives. In contrast, SAS drives 
are built for high-performance, high- 
availability use. They operate at higher 
spindle speeds of 10,000 to 15,000 rpm, 
with compensation for rotational vibra¬ 
tion to ensure data integrity, and are 
built for higher reliability. They are 
designed for environments where data 
transactions are high and data availabil¬ 
ity is essential. 


RAID arrays attached to a SAS con¬ 
troller can be created using either SATA 
or SAS disks, or a combination of both. 
However, the performance and reliability 
of the resulting RAID array is in large 
part a function of the quality of the RAID 
technology and the underlying RAID im¬ 
plementation that is used. 

Depending on the application, a 
SATA disk RAID array can be used as a 
data repository to provide more depend¬ 
ability through intelligent RAID. Simi¬ 
larly, a SAS disk RAID array provides 
more durability and resilience, and higher 
performance, to meet greater business de¬ 
mands for enhanced data availability and 
protection. 


With the union of high-security, cost- 
effective ROMB and the built-in reliabil¬ 
ity and availability features included with 
SAS, system developers can now meet the 
data security requirements of tomorrow 
on the restrictive budgets of today. 4 
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High-Speed Storage Interface 


High-Speed Protocols 

Remove Storage Network 
Bottlenecks 


Enterprises are addressing network infrastructure growth by increasing the 
performance of their storage area networks. Higher-speed server, disk and 
fabric I/O can deliver lower latency and avoid bottlenecks. 


by Abhijit Athavale, 

Xilinx 

W hile most content today is voice- 
and data-intensive, tomorrow’s 
applications will be graphics- 
and video-intensive as the network infra¬ 
structure grows. These applications—in 
addition to existing ones such as asset/in¬ 
ventory management, e-mail, backup and 
recovery, Web content hosting, data min¬ 
ing and ERP—require storage area net¬ 
works (SANs) to service the peak rate of 
demand (Figure 1). 

With the notable exception of Fibre 
Channel (FC), most I/O schemes in the 
storage world today—such as PCI, ATA 
and SCSI—use either source-synchronous 
or system-synchronous clocking. The PCI 
architecture uses system-synchronous 
clocking with a central arbiter that allows 
sharing of a common bus among several 
clients. This has obvious limitations since 
bus bandwidth is not infinite, which in 
turn limits client capabilities. Addition¬ 
ally, this technology does not scale well 
when translated to backplane or box-to- 
box interconnects. 

Source-synchronous technologies 
employing Fow Voltage Differential 
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Signaling (FVDS) I/O were the next ob¬ 
vious choice for designers as they are 
point-to-point interconnects that do not 
have a central arbitration scheme bottle¬ 
neck. However, these schemes suffer from 
the same problem that plagued PCI-based 
schemes when scaling the technology 
across backplanes or box-to-box intercon¬ 
nects. Not only do designers have to run 
tens, or in some cases even hundreds, of 
traces over many inches of FR4, they also 
have to be mindful of the lane-to-lane data 

Server 


and the clock-to-data skew. A bit arriving 
too early or too late can cause serious link 
integrity problems. 

The most logical evolution of system 
I/O now leads us to multi-gigabit serial 
I/O with clock-data recovery (CDR). This 
technology not only reduces the number 
of traces running across boards, it com¬ 
pletely eliminates clock-to-data skew while 
reducing lane-to-lane skew effects for traces 
running tens of inches. FIFOs available 
in multi-gigabit transceivers provide high 

FC Disk Array 
Storage 
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Figure 1 


As the number and type of devices connected to storage area networks 
expands, I/O bandwidth is increasing to avoid bottlenecks and reduce 
latency. 
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Figure 2 


A 64-bit/133 MHz PCI-X connection requires almost 90 pins while 
providing a throughput of 8 Gbits/s, vs. a x8 (8-lane) PCI Express 
link with a data rate of 16 Gbits/s that needs only 32 pins for eight 
2.5 Gbit/s transceivers. 


skew tolerance for channel bonded lanes, 
typically up to 7 ns. The channel bonded 
lanes aggregate individual lane bandwidth 
to provide more channel bandwidth. For 
example, a x8 (8-lane) PCI Express link 
with a data rate of 16 Gbits/s needs eight 
2.5 Gbit/s transceivers, or 32 pins. 

By contrast, a 64-bit/133 MHz PCI-X 
connection would require close to 90 pins 
while providing a throughput of 8 Gbits/s. 
In this case, according to Intel, the board 
area is reduced by 53%, allowing the de¬ 
signer to cut system design costs (Figure 
2). Another significant advantage of this 
technology is the flexibility possible when 
scaling this system. Designers can easily 
partition the system since they are not lim¬ 
ited by the number of signals on the back¬ 
plane. Along with Fibre Channel, which 
has used serial I/O since its inception, 
several standards are also transitioning to 
serial. These include ATA transitioning to 


Serial ATA (SATA), PCI transitioning to 
PCI Express and SCSI transitioning to Se¬ 
rial Attached SCSI (SAS). 

Fibre Channel 

The Fibre Channel specification sup¬ 
ports a variety of speeds and topologies, 
though the majority of FC devices today 
run at 1 Gbit/s or 2 Gbits/s. The new spec¬ 
ification ups this speed to 4 Gbits/s and 
allows operation at 10 Gbits/s. Supported 
FC topologies include point-to-point, ar¬ 
bitrated loop (AL) and fabric, with AL be¬ 
ing the most widely adopted. 

The 4 Gbit/s FC (4GFC) specification 
is fully backward compatible with exist¬ 
ing 1 Gbit/s and 2 Gbit/s equipment and it 
supports AL. Systems based on 4GFC au¬ 
tomatically match the rate of the slowest 
system in the network, enabling the use of 
legacy devices and allowing the speed of 
the fabric to be increased gradually rather 


than forcing a total overhaul of the infra¬ 
structure. 

In addition, there is no significant 
difference in optics or silicon technology 
between 4GFC and 2 Gbit/s FC (2GFC), 
which means that the component price for 
4GFC is comparable to component pric¬ 
ing for 2GFC. The same cables, connec¬ 
tors and software can be used for both, 
allowing enterprises to incrementally 
upgrade their systems to 4GFC. For ex¬ 
ample, a company may choose to upgrade 
only a few servers and disk arrays in the 
critical path to 4GFC, while leaving some 
legacy gear in place. The SAN can now 
perform full rate data transfers when re¬ 
quired, but can de-rate to support lower 
speeds as well. 

The use of 4GFC allows companies 
to run server applications more efficiently, 
perform faster backups and keep mission- 
critical applications up and running. This 
specification matches up very well with the 
newer internal server bus standards such 
as PCI Express, and is capable of trans¬ 
ferring large amounts of data to support 
bandwidth-intensive graphics, video and 
compute applications. Initially, 4GFC was 
designed to connect disk drives to servers, 
but it was also adopted as the SAN switch 
fabric, allowing design and deployment 
of intelligent 4GFC switches. The higher 
per-link bandwidth of 4GFC also allows 
the deployment of a smaller number of 
more powerful servers and higher capac¬ 
ity disk arrays by reducing the number of 
connections and saving costs. Equipment 
based on 4GFC technology is expected to 
continue an aggressive ramp throughout 
this year. 



Fibre Channel (FC/AL) 

Serial ATA 

Serial Attached SCSI 

PCI Express 

Performance 

Full duplex 

Half duplex 

Full duplex with 
link aggregation 
(wide ports) 

Full duplex 

4.0 Gbits/s 
(10 Gbits/s planned) 

1.5 Gbits/s 
(3.0 Gbits/s planned) 

3.0 Gbits/s 
(6.0 Gbits/s planned) 

2.5 Gbits/s 
(5.0 Gbits/s planned) 


> 15m external cable 

1m internal cable 

> 6m external cable 

Add-in card 

Connectivity 

127 devices 

Loop or loop switch 

One device 

(fan-out devices demonstrated) 

> 128 devices 

Expanders (16k Phys. max) 

Switch port- dependent 
(one device per port) 


Fibre Channel only 

SATA only 

SAS and SATA 

PCI Express only 

Driver Model 

Software transparent with SCSI 

Software transparent with parallel ATA 

Software transparent with SCSI 

Software transparent with PCI 


Figure 4 


Comparisons of I/O technologies. 
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SAS Portl SAS Port2 Notch Power 


Figure 3 


Serial Attached SCSI (SAS) 
connector notched to fit SAS 
and Serial ATA (SATA) disks. 


Serial ATA 

Advanced Technology Attachment 
(ATA) is the most dominant disk drive 
interface today. Its simple and cost-effec¬ 
tive implementations, along with several 
enhancements, have helped it serve the 
storage market well. However, ATA also 
has limitations. It requires 5V signaling, 
which is impossible to do at process ge¬ 
ometries of 130 nm and below. The 5V 
signaling also means that the cable length 
must be limited to 45 cm to avoid signal 
integrity problems. The ATA interface 
can only support two drives at most, and 
has no built-in error checking mechanism 
for commands. It also uses a shared archi¬ 
tecture, which limits the bandwidth avail¬ 
able to each drive. 

Serial ATA improves on the ATA spec¬ 
ification by increasing overall bandwidth 
while maintaining complete backward com¬ 
patibility with existing software applications, 
specifically BIOS and OS drivers. Serial I/O 
with 250 mV LVDS signaling is used for the 
new physical layer, which matches with cur¬ 
rent and future silicon process geometries. 
The SATA I specification supports band- 
widths of up to 1.5 Gbits/s while SATA II 
supports up to 3 Gbits/s. 

Serial ATA is mainly intended as an 
in-box interconnect. It significantly re¬ 
duces the tangle of cables from within the 
box, since the connections are now fewer 
and point-to-point. It also allows thin ca¬ 
bles up to lm long with snap-in connec¬ 
tors. In addition to its inherent lower cost, 
enhancements in the SATA standard- 
such as point-to-point connectivity, cyclic 
redundancy check (CRC) on command 
and hot swap/hot plug—have opened up 
new applications, such as backplanes and 
enterprise storage. The SATA I specifica¬ 
tion has already become widely adopted 
while SATA II should start ramping once 
the chipsets supporting it become widely 
available. 


Serial Attached SCSI 

Another popular disk interface used 
to connect disk and tape drives, printers 
and optical storage is SCSI. Unlike ATA, 
SCSI can also transfer 8- or 16-bit data 
to external devices using cable. It enables 
block data transfers and has a command 
structure shared with Fibre Channel. 

The latest SCSI specification calls 
for LVDS signaling to deliver a through¬ 
put of 320 Mbytes/s. But SCSI is still not 
the best option when a large number of 


devices need to be connected. Its shared 
arbitration structure does not lend itself 
well to hot-swapping drives. It can only 
support up to 15 drives with one bus. 
Adding more drives means adding more 
SCSI host adapters and multiple strings of 
drives. This implementation does not lead 
to high reliability in storage arrays. 

Serial Attached SCSI is designed 
to remove the limitations to the SCSI 
specification. It is a point-to-point, full 
duplex, serial interface. Since it now 
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uses SATA-compatible cables and con- 
nectors—the new SAS connector is a 
SATA connector with additional signal 
lines—it lends itself well to high volume 
markets (Figure 3). 

The interconnection and drive expan¬ 
sion scheme is also much simplified: a 
single SAS domain can now support up to 
16,382 devices. Serial Attached SCSI sup¬ 
ports SCSI, SATA and a SCSI management 
protocol, in essence allowing the design of 
a single interface that can support both 
SATA and SAS drives. It also provides 
redundancy options for high-availability 


storage arrays. Drives and systems based 
on SAS are expected to ramp up in 2006. 

PCI Express 

PCI Express is the next generation of 
the ubiquitous PCI bus. It uses serial I/O 
technology and is fully backward com¬ 
patible with the PCI driver and software 
model. It can scale from a 1-lane (xl) to a 
32-lane (x32) system with each lane deliv¬ 
ering a raw line rate of 2.5 Gbits/s before 
line coding. 

The PCI Express specification is a 
three-layer specification including physical, 
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data-link and transaction layers. The physi¬ 
cal layer deals with electrical and mechani¬ 
cal requirements and is well defined to ac¬ 
commodate different form-factors, such 
as ExpressCard, Minicard and Express 
Module. It can also support future higher 
speeds. The data-link layer handles reli¬ 
ability and is responsible for delivering 
good packets using CRC. The transaction 
layer is where the protocol is executed and 
data can be moved as packets. 

PCI Express also has some enhance¬ 
ments such as virtual channels and classes 
of traffic that allow user applications to 
set priorities for which packet should get 
transferred first. PCI Express also defines 
different packet sizes with a maximum 
payload of 4 Kbytes. 

PCI Express is mainly an in-box 
server interconnect. While not a true stor¬ 
age protocol like Fibre Channel, SATA 
and SAS, it matches up well with those in 
performance, and will be the primary pro¬ 
tocol used by servers and PCs to commu¬ 
nicate between the host processor and pe¬ 
ripherals. The PCI Express product ramp 
has already begun. PCs with multiple PCI 
Express slots can be purchased today for 
less than $1,000. According to Intel and 
Crystal Cube Consulting, by 2007 80% of 
existing PCI ports will be replaced by PCI 
Express ports. 

New applications and the always- 
expanding desire for information are in¬ 
creasing demands on storage networks to 
provide improved performance that can 
supply data at peak rates. The industry has 
responded to the challenge by defining 
new protocols that remove inefficiencies 
and bottlenecks in SANs (Figure 4). The 
common thread among all of these new 
protocols is the fact that they are back¬ 
ward compatible with software and driv¬ 
ers written for older protocols, making the 
transition much easier and less expensive 
for most companies, d 
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ATCA and AMC 


Standards-Based Platforms: 

The Path to ATCA 
Applications 

The relentless miniaturization of electronics components is encroaching 
on the telecom space at an astounding pace. The result is a shift from 
expensive, custom, proprietaiy systems with long development cycles 
toward low-eost, standards-based systems built from highly commoditized, 
off-the-shelf components. 


by Stan McClellan, Paul Fleming and 
David Dean Smith, Hewlett-Packard 


T he problem of delivering and sup¬ 
porting complex network elements 
for the telecom market has fallen 
naturally into two distinct development 
phases: system integration and element 
creation. In the system integration phase, 
industry-standard subsystems are inte¬ 
grated into a functional “base platform,” 
which consists of arbitrary hardware, 
operating system and manageability ele¬ 
ments. These base platforms are then used 
in the element creation phase as the start¬ 
ing point for the rendering of a completed 
network element. By properly leveraging 
the first phase of this process, Network 
Equipment and Service Providers (NEPs 
and NSPs) will be able to realize the ben¬ 
efits of a configurable, common platform 
offering, which reduces time-to-market 
via commoditization, maximizes interop¬ 
erability via industry standards and opti¬ 
mizes net revenue. 

With this two-stage process in mind, 
the concept of an optimal “base platform” 
built from ATCA components consists of 
several functional areas: 
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PICMG 3.0-compliant shelf, includ¬ 
ing power entry modules, cooling in¬ 
frastructure, backplane and integrated 
shelf management. 

PICMG 3.1-compliant transport infra¬ 
structure. The transport infrastructure 


of the shelf (redundant hub blades) 
supports the PICMG 3.0 Base Inter¬ 
face as the high-bandwidth shelf con¬ 
trol plane. In addition, multiple bearer- 
plane networks are supported via 
the extended Fabric Interface, which 


Characteristic 

14-slot NEBS or 16-slot ETSI shelf 

Height/Width/Depth 

575 mm/439 mm/507 mm (16-slot ETSI shelf is 533 mm wide) 

Rack Mount 

2-post or 4-post 

Power Entry 

Dual hot-swap with filtered input & segmented output 

Backplane Topology 

Dual-star. Both Base and Fabric Interfaces aggregate in redundant 
hub slots 

IPMB Topology 

Radial 

Shelf Management 

Dual hot-swap, located in the air intake path 

Cable Trays 

Front & rear 

Fan Trays 

Hot-swappable, high-performance with 3 fans per tray 

Cooling Efficiency 

Sufficient for 200W payload with a 20W RTM 

Air Filter 

Filter on inlet with filter presence sensor 

Alarm Panel 

Alarm interface and serial console for shelf manager 


Figure 1 


Optimal ATCA shelf characteristics for HP’s AOTP platform. 
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ATCA Fan Module Performance 
per cell 



mock cell impedance 
Shelf A 

Best performing 14-slot shelf 


• CPU blade impedance 
— Shelf B 

Best performing 16-slot shelf 


Figure 2 


Data from airflow test chamber. HP is planning for thermal headroom 


implements the dual gigabit Ethernet 
fabric of PICMG 3.1, channel options 
1, 2 and 3. 

• Node blades. The node or payload 
components of the shelf are composed 
around a carefully defined set of gen¬ 
eral-purpose processor blades. These 
blades leverage a common architec¬ 
ture that supports a range of compute, 
memory and Advanced Mezzanine 
Card (AMC) expansion options. 

• Payload blade expansion options. Pe¬ 
ripheral expansion devices include 
AMC-based implementations of com¬ 
mon I/O interfaces. 

The themes common among all sub¬ 
systems are the notions of compliance 
with dominant industry standards and po¬ 
tential for commonality and re-use among 
multiple applications and customers. 

Basic System and Control 
Plane—PICMG 3.0 

The fundamental component in a 
family of ATCA solutions is the shelf. To 
have widest applicability, the shelf must 
be compliant with the form-factor, elec¬ 
trical, environmental and other system 


requirements of both NEBS2000 and 
ETSI. Additionally, the shelf must pres¬ 
ent ease-of-use features such as front/rear 
cable trays, and serviceability features 
such as easily accessible fan and power 
modules, input air filters, shelf status in¬ 
dicators and dry-contact telecom alarm 
inputs/outputs. HP’s evolving Advanced 
Open Telecom Platform (AOTP) strategy 
is based on ATCA standards. The mini¬ 
mal characteristics for the “base platform” 
shelf are shown in Figure 1. 

One of the most difficult aspects of 
ATCA shelf design is the thermal effi¬ 
ciency of the cooling infrastructure. Sub¬ 
tle optimizations in the design of the shelf 
can be critical in the availability and life 
cycle management of the payload compo¬ 
nents. The ATCA shelf assemblies avail¬ 
able in today’s ATCA ecosystem present 
a mix of thermal solutions as well as ver¬ 
tical heights. These thermal approaches 
include positive pressure systems with fan 
trays on the bottom of the shelf, negative 
pressure systems with fan trays on the top 
of the shelf, and hybrid systems with fan 
trays on the bottom and top of the shelf. 
In evaluating the available shelves, the 
ambiguities of payload cooling needs and 


the performance of contending shelf de¬ 
signs led us to a clear requirement for a 
height of at least 13U (575 mm) for ver¬ 
tically mounted blades. Shelves with less 
than 13U height don’t provide enough 
headroom for the thermal requirements of 
future payload devices. 

The physics of the airflow through 
the shelf defines the ability of the system 
to adequately cool the payload electronics. 
In general, the denser the payload device, 
the more difficult it is for air to circu¬ 
late through the electronic components 
and remove the heat being generated. 
To establish a performance baseline for 
various shelves, we used Computational 
Fluid Dynamics (CFD) models to verify 
the airflow and thermal requirements of 
combinations of payload devices. In this 
approach a guiding assumption is that 
blades/processors will become hotter and 
denser over time. 

To compare theoretical and actual 
performance, we built a complete worst- 
case scenario for payload configurations 
and tested it in an airflow chamber. This 
payload configuration was chosen to 
simulate the most problematic case for 
physical impedance of airflow: an AMC 
carrier blade with multiple installed AMC 
devices. By plotting air-pressure vs. air¬ 
flow curves for each possible shelf con¬ 
figuration, and by taking into account the 
available fan trays and maximum payload 
blades per shelf, we were able to deter¬ 
mine the thermal management capability 
for available shelves under the worst-case 
scenario. 

Figure 2 shows curves of fan module 
performance in cubic ft/meter (CFM) for 
two eco-system shelf configurations (A 
and B) as well as the best performing 14- 
slot and 16-slot shelf configurations. The 
data represents a single slot in the shelf 
at a specific power load to the fan trays 
(in this case, full power and full blower 
speed). The shelf configurations are plot¬ 
ted against the airflow impedance curve 
for an actual payload device. The cross¬ 
ing point of the shelf and blade curves 
represents the amount of air that will flow 
through the blade (the operating point). 

To extend this approach to potential 
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ATCA CPU Thermal Testing 

Processor and both HDD AMCs running at max Power 



Figure 3 


Thermal data from a 16-slot shelf and functional CPU Payload Blades 
(one processor/2 Storage AMCs). 
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Figure 4 


Transport subsystem showing support for PICMG 3.1 channel 
options 2 and 3. 


customer configurations, we again used 
airflow chamber testing to verify thermal 
data supplied by blade manufacturers. Fig¬ 
ure 3 shows a representative result from 
airflow testing of a fully loaded, high-per¬ 
formance shelf. In all cases, system-level 
validation was performed for the follow¬ 
ing cases: 

• Normal Mode - All Fan Trays func¬ 
tional 

• Failure Mode - One blower fail 

• Failure Mode - Fan Tray removed for 
service 

• Filler panel removed - 3 minute max 
time frame 

The acoustic requirements for ATCA 
are challenging given the amount of air¬ 
flow that HP considers necessary to suf¬ 
ficiently cool the AMC-based payload 
blades. The best acoustic design is to bury 
the fan trays toward the back of the shelf 
in order to attenuate the noise to the front. 
The design goal is to keep the acoustic 
level below 60 dBA during normal operat¬ 
ing conditions (0°-40°C) and only exceed 
this level when the ambient temperature at 
the inlet is between 40° to 55°C, or when 
there is a fan tray failure that forces the 
system to run the fan trays at full speed. 
In these cases, temperature excursions 
and fan tray failures are short-term events 
that can temporarily exceed the 60 dBA 
acoustic limit. 

Bearer Plane Subsystem 

HP’s vision of today’s optimal ATCA 
platform implements a dual-star architec¬ 
ture, with each node slot having redundant 
connections to the hub slots. The blades 
resident in the hub slots support the fea¬ 
tures required by PICMG 3.0, including 
the Ethernet base interface and manage¬ 
ability interfaces. To implement a first- 
generation fabric interface, the switch 
blades also support multiple Gigabit Eth¬ 
ernet transports compliant with a sub¬ 
set of PICMG 3.1 channel options. The 
cost-effectiveness of Ethernet as a con¬ 
trol-plane and bearer-plane interconnect is 
evident in this system architecture, which 
is illustrated in Figure 4. An interesting 
modification of this architecture is the use 
of a dual-dual-star backplane where a sec¬ 
ondary pair of hub blades can be installed 
to enable in situ upgrades of the transport 
subsystem of the shelf. 
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disk and backplane/RTM Ethernet 


(b) An embeddded protocol or media 
gateway with network interface 



(c) A network-attached storage server 
with RAID5 local SAS disks 




(e) Multiple independent embedded 
processing nodes 


Figure 5 


Some representative usage models for the AMC carrier board. 


The hub blades provide the trans¬ 
port subsystem for nodes in the shelf. For 
highly available deployments, the switch 
complex must provide a failover mecha¬ 
nism that preserves the context for bearer 
traffic. A cost-effective implementation of 
this failover link uses a point-to-point 10 
Gigabit link across the update channel be¬ 
tween switch slots. 

The base interface implements 
10/100/1000 Mbit/s Ethernet as specified 
in PICMG 3.0. The switch for the base 
interface is supplied on the hub blades in 
the dual-star architecture, or on the funda¬ 
mental hub blades in the dual-dual-star ar¬ 
chitecture. All ports of the base switch are 
able to simultaneously forward Ethernet 
frames at full line rate, and support link 
aggregation via IEEE 802.3ad to enable 
higher bandwidth base ingress/egress. In 
addition to fairly common Ethernet fea¬ 
tures such as hardware-based flow control 
via IEEE 802.3x and support for “jumbo 
frames” up to 9K bytes in length, the base 
switch also supports integrated switch/ 
routing for IPv4, which maximizes packet 
throughput via hardware acceleration of 
certain forwarding functions. 

The fabric interface implements 1000 
Mbit/s Ethernet as specified in PICMG 
3.1, channel options 1, 2 and 3. The switch 
for the fabric interface is supplied on the 
hub blades along with the base interface 
switch. The fabric switch provides Ether¬ 
net ports to support at least two in-shelf 
fabric interface ports for all payload slots 
(PICMG 3.1, channel option 2), as well as 
several ports for ingress/egress. The fab¬ 
ric switch provides additional capacity by 
routing two additional Ethernet ports to 
each of several payload slots (PICMG 3.1, 
channel option 3), as shown in Figure 4. 
This configuration allows for a very effec¬ 
tive compromise between the base-only 
implementations of early ATCA systems 
and future implementations of 10 Giga¬ 
bit (XAUI) fabric and other PICMG 3.x 
bearer plane technologies. Similar to the 
base switch, the fabric switch has com¬ 
mon Ethernet features and optimizations. 

The concept of Quality of Service 
(QoS) is very important for the telecom 
environment, particularly in the context 
of isochronous media streams. Unfortu¬ 
nately, the Ethernet-based link-layer dis¬ 
cussed here (layer 2) does not have mech¬ 
anisms to enable full per-stream QoS for 


all possible transport requirements. In ad¬ 
dition, the likely use of this ATCA infra¬ 
structure with IP-based application traffic 
complicates a full QoS implementation 
due to a requirement for synchronization 
between existing Ethernet and standard 
network-layer (layer 3) mechanisms such 
as DiffServ. 

Fortunately, a Class-of-Service (CoS) 
implementation provides sufficient granu¬ 
larity for many cases to ensure the adequate 
treatment of aggregated payload streams. 
Since the proposed transport subsystem 
for ATCA is based purely on Ethernet, 
the implementation of CoS mechanisms 
is exposed at layer 2 via compliance with 
relevant subsections of IEEE 802.1. Both 
the base and fabric switches implement 
CoS functions such as frame prioritization 
(802.1p) and VLAN tags for grouping and 
isolation of bearer traffic (802. lq). 


Payload Components 

The ATCA base platform specifies a 
holistically optimized family of payload 
blades to provide general-purpose pro¬ 
cessing, basic storage and common I/O re¬ 
quirements. At the low end of the process¬ 
ing spectrum, the AOTP base platform 
specifies an AMC carrier blade. This util¬ 
ity blade allows for a wide variety of con¬ 
figurations via four AMC slots compatible 
with the .0/.1/.2/.3 series of the PICMG 
AMC specifications. In particular, each 
of the AMC slots exposes multiple inter¬ 
faces to the carrier and backplane, includ¬ 
ing PCI Express (x4 lanes, or 8 Gigabits/s 
each direction) and SAS/SATA storage 
interfaces. The carrier blade also supports 
dual Gigabit Ethernet links for PICMG 
3.1 channel option 2 at the blade level, as 
well as managing IPMI interactions for 
the mezzanine cards. 


November 2005 FMH33 










































































It’s a Smooth Landing in ATCA™ 



with AudioCodes on Board 


With more than a decade of expertise in VoIP, AudioCodes 
has deployed its feature-rich products in networks of leading 
customers around the world. Offering a wide variety of field- 
proven products, AudioCodes complies with the rapidly 
evolving international market standards and requirements. 
When selecting the right building blocks for your ATCA™ 
system, you can rely on AudioCodes for interoperability, 
scalability, responsiveness and reliability. 

Leveraging on our sound track record, AudioCodes introduces 

the TP-12610 ATCA™ VoIP Media Gateway Board. Designed 
for high density applications, the TP-12610 supports up to 
4.000 LBR channels and an array of PSTN and networking 
interfaces, all on a single blade. 

For more Information call +1-408-577-0488 
or email VolPsolutions@audiocodes.com 


AdvancedTCA® I ^AudioCodes 

www.audlococfes.com 


SolutionsEngineering 


For stand-alone configura¬ 
tions, a PCI Express bridge is inte¬ 
grated on the carrier blade so that 
host devices such as a processor AMC 
(PrAMC), can enslave other AMC de¬ 
vices. For example, a PrAMC in an 
AMC slot can act as the host for I/O 
expansion devices, special-purpose 
processing devices, or storage devices 
in neighboring AMC slots. To enable lo¬ 
calized storage, pairs of AMC slots are 
directly interconnected by SAS/SATA 
ports so that AMC disks can be used as 
local storage for PrAMCs. A number of 
representative configurations are shown 
in Figure 5. 

For general-purpose computing 
needs, the Intel processor architecture 
presents a well-resolved ecosystem for 
system configurations, with performance 
reaching well into multi-GHz clock 
rates. So, the AOTP base platform fam¬ 
ily of server blades implements IA-32 and 
I A-64 CPU architectures with varying 
memory capacity, AMC slots and pro¬ 
cessor configurations. The AMC slots 
can be used for a number of peripherals 
supporting PCI Express. In all cases, the 
payload blades support common AMCs 
and/or rear transition modules (RTMs) 
through which storage and I/O function¬ 
ality can be added and the system tailored 
for specific applications. 

The common components of the ar¬ 
chitecture include a CPU complex hous¬ 
ing one or two sockets. Supporting these 
processors are high-performance chipsets 
and peripherals providing native PCI Ex¬ 
press as well as several redundant Ethernet 
interfaces for Base and PICMG 3.1 fabric. 
Each blade in the family supports a range 
of memory options, with maximum per- 
blade configurations reaching 32 Gbytes 
or more. 

These blades share a common funda¬ 
mental architecture as depicted in Figure 
6. Standard equipment for all server blades 
also includes an integrated SAS/SATA 
controller with interfaces mapped to AMC 
slots, and the RTM, telecom clock inputs 
and PCI Express extended to the back¬ 
plane to enable multi-slot, highly flexible 
blade configurations. These features are 
in addition to the usual I/O mappings to 
the faceplate (serial, USB 2.0 ports), the 
indicators and other features required by 
the PICMG 3.0 specification, and the mul¬ 
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Figure 6 


Simplified block diagram of common Server blade architecture. 


titude of RTM possibilities available for 
each server configuration. 

Since development of semi-custom 
devices requires a substantial investment, 
the low unit volumes in the ATCA market 
may allow only a few viable niches. To ad¬ 
dress more intensive computing require¬ 
ments, multi-core processors are already 
becoming commonplace. In addition to 
requiring less power—which is a signifi¬ 
cant issue in dense telecommunications 
deployments—multi-core processors en¬ 
able efficient, dynamic hardware-based 
resource allocation for specific tasks. A 
common term for this approach is “on- 
loading.” 

As indicated previously, the concept 
of a holistically designed payload blade 
family for such a base platform is com¬ 
patible with the current versions of the 
AMC specifications (.0/.1/.2/.3). So AMC 
modules are the preferred mechanism for 
I/O expansion and other peripheral de¬ 
vices. The fundamental set of AMC de¬ 


vices preferred for AOTP are those preva¬ 
lent in telecom networks, such as Gigabit 
Ethernet interfaces, storage modules and 
common signaling (El/Tl) and transport 
interfaces (STM1/OC3). Other peripheral 
functions that have special-purpose pro¬ 
cessors, application-specific uses or much 
narrower potential adoption are viable op¬ 
tions for the AOTP program, but are han¬ 
dled in a “semi-custom” fashion, d 
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ATCA and AMC 


Building Scalability into 
ATCA and AMC 


Building a scalable network element using ATCA as the hardware 
platform can take advantage of the Forwarding and Control Engine 
Separation protocol to make networks scale from blades to shelves to 
multiple chassis with load balancing and upgradeability. 

by Alan Deikman 
ZNYX Networks 


O ne of the favorite marketing terms 
associated with ATCA is “scalabil¬ 
ity.” Even though this feature is 
highly thought of and almost universally 
sought after, actual open-standard imple¬ 
mentations that are truly scalable are rela¬ 
tively rare. The ideal of scalability is that if 
a single payload blade such as an SBC, net¬ 
work processor, DSP Resource or storage 
unit can handle a workload X , then all that 
is needed to handle a system that requires 
up to nX units of capacity is to simply in¬ 
stall n payload blades or “field replaceable 
units” (FRUs) of the required type. Further, 
this often implies that the system can be 
augmented (and less frequently downsized) 
while it is operating. 

Making this work in practice takes 
considerable skill and sophistication in the 
architecture for both hardware and soft¬ 
ware. From an architectural point of view, 
the AMC series of specifications are nearly 
identical to ATCA except that they have a 
higher granularity. For example, an SBC 
on a half-height, single-wide AMC card 
potentially has all the same characteristics 
and features of an SBC implemented on an 
ATCA blade, except it will have smaller ca¬ 
pacity, performance and unit cost. The con¬ 
cepts presented here apply equally to AMC 
as they do to ATCA. 



Get Connected with companies 
mentioned in this article. 
www.rtcmagazine.com/getconnected 


ATCA derives its scalability fea¬ 
tures from the use of LAN technology. 
At the block-diagram level there is no 
difference between a network connec¬ 
tion via a detachable cable or one estab¬ 
lished via a backplane. What ATCA has 
that is not generally present in a conven¬ 
tionally cabled LAN is a standardized 
multi-plane structure including the fol¬ 
lowing elements: 

• An PC-based IPMI management system 

• A Base Interface that can be dedicated 
to intra-device control functions 

• A Fabric Interface (as described in one 


of the subsidiary specifications PICMG 
3.1, 3.2 etc.) that can be dedicated to 
payload 

Although the structure imposed by 
ATCA is not essential for building scalable 
devices, it does make the end result more 
marketable at the expense of circumscrib¬ 
ing an upper limit. A single ATCA chas¬ 
sis has a maximum of 14 payload blades, 
where a LAN is theoretically unlimited. 
What makes ATCA attractive is that the 
familiar hot-swap card design, the manage¬ 
ment infrastructure and the multi-vendor 
support help make the concept more mar- 



Exterior Network Connections 


Figure 1 


ForCES Model: The variable number of FEs implement per-packet 
operations under the control of the CEs. 
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Network Connections 


Figure 2 


One mapping of ForCES on ATCA: the FEs and CEs may appear in any 
combination of payload blades and the ATCA Flub devices provide 
interconnect. 


ketable into telco and enterprise markets. 

The ForCES Model 

Independently from PICMG’s ATCA, 
a considerable effort over several years has 
been put forth by a working group within 
the Internet Engineering Task Force 
(IETF) to standardize a protocol that en¬ 
ables a scalable network element model. 
The protocol is called ForCES, an acro¬ 
nym for Forwarding and Control Engine 
Separation. 

The concept of ForCES is simple on 
its surface. The specification defines any 


“network element” (which is the type of de¬ 
vice discussed in this article) to consist of at 
least one “control engine” (CE) that controls at 
least one “forwarding engine” (FE) as shown 
in Figure 1. To achieve a desired level of fault 
tolerance, capacity or features, any number of 
CEs and FEs can be used. 

The role of the FE is to process pack¬ 
ets in any manner desired, and they may be 
implemented with any available technology 
from all software running on a general-pur¬ 
pose processor, network processor, DSP 
device or an ASIC. The internal operation 
of an FE is fully transparent and abstracted 


Network Connections 



ATCA hubs act as load-balancing FEs. 


from the outside world, making it possible to 
freely exchange different implementations 
of FEs that perform the same function with 
different price/performance characteristics. 
For example, a basic service implemented 
with a general-purpose processor might be 
able to handle 1,000 connections per sec¬ 
ond, while a network processor might be 
able to handle 10,000 connections at five 
times the unit cost. The ForCES-defined 
abstracting of FEs also raises the possibil¬ 
ity of multiple-vendor network elements, 
which is a similar goal of ATCA. 

Some FEs may be “line cards,” which 
connect to the network, and some may be 
“service cards” that have no external con¬ 
nection. Operations performed by an FE 
may include any packet-by-packet opera¬ 
tion, from simple switching operations on 
any layer through complex and stateful 
packet modifications. 

A packet can transit any number of FEs 
as required to perform the complete func¬ 
tion of the network element, which allows 
applications of any level of complexity. For 
this reason the FEs generally require inter¬ 
connections (not shown in Figure 1) that 
may be any type of topology or network 
desired. 

The CE components contain the 
software that controls all the FEs via the 
ForCES protocol. Operations include the 
configuration and setting of tables and the 
monitoring of status. If the network ele¬ 
ment supports management protocols such 
as SNMP or an HTTP interface, this func¬ 
tion is supported by the CE either directly 
or indirectly. Although only one CE is nec¬ 
essary, more than one is generally supplied 
to provide redundancy or scalability. 

To apply a ForCES model to ATCA, 
each of the CE/FE components in the design 
can be mapped to a given ATCA or AMC 
FRU. Some FRUs may be able to handle 
a mix of CE/FE functions. One such pos¬ 
sible mapping as related to ATCA is shown 
in Figure 2. Many variances of this model 
can be constructed. For example, the FE to 
FE interconnect could be implemented as 
a VEAN on the Base Interface in a chassis 
without a fabric interface, or ingress/egress 
ports may be connected to the hubs instead 
of the FEs. 

The example mapping in Figure 2 has 
the advantage of being highly intuitive, and 
can be easily extended to include AMC 
components in addition to or instead of 
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ATCA FRUs. Each FRU (other than the 
hubs) is a CE or an FE. Each FE has po¬ 
tentially equal access to the backplane and 
can provide its share of capacity. If the CE 
implementation is sophisticated enough, a 
technician can add or remove FEs up to the 
capacity of the backplane without having to 
worry about software configuration. The 
ForCES specifications allow the CEs to 
have control over the FEs, which includes 
all loading of configurations and tables. 

In conventional (non-ForCES) designs, 
where the CE and FE are integrated on a 
single FRU, this type of modularity is much 
more difficult to achieve, if not impossible. 
It is also possible to have more than one 
physical network element hosted within a 
single ATCA chassis to achieve multiple- 
tenancy, or to have a network element span 
more than one ATCA chassis to implement 
open-ended scalability. In such cases the 
concepts presented here apply equally as 
long as the interconnections are extended 
appropriately. 

The Load Balancing Problem 

A key problem to be solved in the scal¬ 
able network element is how to route in¬ 
coming packets to the appropriate FEs for 
processing in such a way that the process¬ 
ing load is distributed among them. With¬ 
out this facility the system will be limited 
to the capacity of one blade. 

In the design in Figure 2, the network con¬ 
nections outside of the device are all located 
on the FE cards. This is the model to use if 
one of the functions of the network element is 
to concentrate LAN/WAN connections. For 
example, each ATCA payload could add 24 
Gigabit Ethernet ports or an OC-192 termina¬ 
tion. This type of network element is scalable 
by ingress/egress capacity. 

Another category of network element 
requires scalability in the dimensions of 
computational power or storage. A security 
device such as an IP security protocol (IP- 
SEC) gateway is a good example. In this 
case the limitation of overall throughput is 
something less than a fixed set of ingress/ 
egress ports. A single encryption/decryp¬ 
tion FRU may be able to handle on the or¬ 
der of 200,000 packets-per-second, where 
a Gigabit Ethernet link is able to receive 
more than 1.4M PPS. 

In this case it is desirable to be able 
to provide only the amount of hardware 
needed for the given traffic load, which is 


often considerably less than the bandwidth 
of the ingress/egress network link. To do 
this, a load-balancing FE is introduced at 
the ATCA hub sites. Since the ATCA hubs 
have the maximum bandwidth available to 
the chassis, they are in the best position, 
as shown in Figure 3, to distribute packets 
among the FEs that have encryption/de¬ 
cryption resources. 

ForCES makes this approach workable 
by treating the ATCA Hub devices as load 
balancing FEs under the control of the active 
CE. Some PICMG 3.1 ATCA hub devices on 
the market have a capability of performing 
this distribution at line rate with ASIC sili¬ 
con, avoiding the need to have general-pur¬ 
pose processors or network processors allo¬ 
cated to this task. This avoids the expense 
of the extra FRUs to perform that task, and 
simplifies the management of cabling since 
only one redundant “fat pipe” is required re¬ 
gardless of the scale of the system. 

While we have focused primarily on 
scalability, other benefits are also attained 
by applying the ForCES architecture. Fault 
tolerance can, for example, be implemented 
with redundant CEs and FEs to construct 


a network element with no single point of 
failure, and to automatically route traf¬ 
fic around elements taken out of service. 
Additionally, the CE in the ForCES archi¬ 
tecture has access to all the management 
information in the network element. This 
single point of management makes it pos¬ 
sible to provide management service to any 
outside entity. 

The OEM can be given more freedom 
from several directions. First, when sup¬ 
ported by the CE software, new FE devices 
can be added that, in addition to added 
capacity, bring new features to a network 
element thus protecting the overall system 
against obsolescence. Additionally, with 
adherence to the specifications, it is pos¬ 
sible to construct systems with different 
vendors providing different elements, free¬ 
ing the OEM to make more choices, d 
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ATCA and AMC 


AMC Provides Foundation 
for ATCA Expansion and 
MicroTCA Chassis 

Whether complementing an ATCA carrier board as mezzanines or assembled 
into card cages as MicroTCA, the AdvaneedlVIC card offers a wide range of 
configuration options for assembling high-performance systems. 


by Jeff Durst 

Artesyn Communication Products 


I n early 2004, PICMG approved the 
initial specification for a new packet- 
based, hot-swappable mezzanine inter¬ 
face for AdvancedTCA blades known as 
AdvancedMC (AMC). Featuring a large 
form-factor, high power handling capabil¬ 
ity and integrated system management, 
the new interface provided a modular 
framework for building high-availability 
telecom systems that could be designed, 
manufactured, scaled, upgraded and ser¬ 
viced at a much lower cost. 


Though still in its infancy, AMC 
has already taken center stage as the 
foundation for a new, small form-factor 
telecom platform. Known as MicroTCA, 
the new platform essentially replicates 
the AdvancedTCA carrier environment, 
enabling OEMs to utilize AMC modules 
directly in a range of chassis (i.e., shelf, 
cube) without the need for an ATCA car¬ 
rier. PICMG performed the first physical 
MicroTCA demonstration at SuperComm 
this summer using a 4U system equipped 


with five Pentium M-based modules, 
which simulated a wireless application 
servicing subscribers (Figure 1). Final ap¬ 
proval for the specification is expected in 
April 2006. 

AMC As a Mezzanine Interface 

AMC provides a number of features 
that make it appropriate as a mezzanine 
expansion interface for ATCA blades. 
Topping the list is hot-swappability. Un¬ 
like traditional mezzanine cards such 
as PMC, which must be bolted on at the 
factory, AMC cards can be inserted or 
removed from live systems as they oper¬ 
ate in the field. This field upgradeability 
greatly enhances overall system flexibil¬ 
ity, enabling service providers to service, 
scale and upgrade their ATCA systems at 
the module level, eliminating the need to 
take entire ATCA blades off line. 

Another key feature that makes AMC 
attractive to designers of network equip¬ 
ment is its high-speed serial packet inter¬ 
face. AMC essentially extends the ATCA 
fabric (10 Gbits/s per link) to individual 
modules, providing up to 21 I/O channels 
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with a bandwidth of 10 Gbits/s per chan¬ 
nel. As with ATCA, the AMC packet in¬ 
terface is also protocol-agnostic, enabling 
it to support a variety of packet-oriented 
protocols, including Ethernet, PCI Ex¬ 
press and serial Rapid I/O. 

AMC’s large footprint (up to 148.5 x 
180.6 x 28.95 mm) and high power handling 
capability (up to 60W) also make it very at¬ 
tractive to network equipment providers, 
enabling them to implement complex func¬ 
tions at the module level. Equally important 
is AMC’s overall mechanical flexibility, 
which offers a range of form-factors that 


enable designers to partition their blades in 
a way that maximizes scalability, upgrade- 
ability and field serviceability. 

AMC supports four module sizes: 
half-height single width, half-height 
double width and a full-height version of 
both of these. The modules have escalat¬ 
ing power limits of 20W for the smallest 
half-height single width to 60W for the 
largest module, which is double-wide full 
height. The AMC spec provides guide¬ 
lines for building three types of ATCA 
carrier boards—short, long and hybrid. 
The short carrier has bays for up to eight 


modules. The long carriers provide bays 
for up to four modules. The hybrid carrier 
provides for portions of either long, short 
or no module bays along the faceplate of 
the board (Figure 2). 

By mixing and matching short, long 
and hybrid carriers with various mod¬ 
ules, designers can readily configure their 
ATCA system with the desired degree of 
granularity. An ATCA shelf, which typi¬ 
cally holds up to 12 ATCA carriers, popu¬ 
lated with long carriers for example, can 
hold up to 48 AMC modules. This high 
degree of modularity greatly enhances 
availability by enabling designers to up¬ 
grade, scale and service their systems at a 
very fine-grained level. 

Another feature that makes Ad- 
vancedMCs very attractive to TEMs and 
service providers is its integrated system 
management. AMC modules support the 
same PC-based Integrated Peripheral 
Management Interface (IPMI) and shelf 
management capabilities as ATCA. The 
IPMI module management controller 
(MMC) on each AMC module gathers in¬ 
formation on parameters like temperature 
and voltage that are considered vital to the 
module’s normal operation, and conveys 
them to shelf management, which takes 
the necessary system actions to ensure 
proper operation and report boards that 
need attention or servicing. This ability 
to pinpoint faults and take corrective ac¬ 
tion at the module level greatly enhances 
availability and serviceability. 

AMC As a Stand-Alone Chassis 
Blade 

The same capabilities that make AMC 
attractive as a mezzanine architecture 
make it equally attractive as a blade-level 
specification for MicroTCA systems. Its 
hot-swappability enhances availability by 
enabling live systems to be serviced and 
upgraded in the field. Its large form-factor 
and high power capability make it ideal 
for implementing complex functions. Its 
high-bandwidth, protocol-agnostic packet 
interface provides an ideal interconnect 
for linking multiple modules in a chassis. 
Its IPMI interface facilitates centralized, 
fine-grain monitoring and control. And 
its flexible form-factor makes it possible 
to create mechanical MicroTCA chassis 
packaging options that are optimized for 
particular applications. 


300 mm 



y 4U / 7” 


Figure 3 


The shelf version of MicroTCA chassis can handle a wide range of AMC 
card sizes for various applications. 
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MicroTCA is in some respects a re¬ 
packaging of modular ATCA/AMC blades 
for small form-factor, cost-sensitive appli¬ 
cations. ATCA’s large form-factor, though 
suitable for building high-density central 
office telecom infrastructure equipment, 
precludes its use in many outside plant and 
enterprise applications with tight size con¬ 
straints. High cost also hampers the use 
of ATCA solutions in many outside plant, 
enterprise and customer premises applica¬ 
tions. ATCA carriers equipped with AMC 
modules have an added cost premium, as 
the carriers must be equipped with expen¬ 
sive card-cage-style connectors in order to 
house field replaceable AMC modules. 

MicroTCA reduces size and cost by 
eliminating the ATCA carrier and en¬ 
abling AMC modules to be used directly 
in a variety of compact, low-cost enclo¬ 
sures, from stand-alone pico cells, to 
standard rackmount systems. The OEM 
production price for a baseline MicroTCA 
system, including a MicroTCA chassis, 
switching hub and power module, is pro¬ 
jected to range from $1,500 to $2,000. 

To accommodate a broad range of 
applications, MicroTCA is designed with 
scalability in mind. In addition to its sca¬ 
lable packaging and power options, Mi¬ 
croTCA provides scaleable bandwidth of 
1-40 Gbits/s. The MicroTCA architecture 
also provides a scaleable level of avail¬ 
ability ranging from three nines (.999) to 
five nines (.99999). 

MicroTCA’s compact format, low 
cost and low power consumption make it 
a perfect complement to ATCA for small 
form-factor central office and outside plant 
applications like wireless base stations, 
digital loop carriers, optical ADMs and 
Fiber to the Curb optical network units. 
Some also see a role for MicroTCA in en¬ 
terprise networking applications such as 
workgroup routers, modular servers and 
SAN storage boxes. 

Virtual Carrier Environment 

A MicroTCA enclosure acts as a vir¬ 
tual carrier, emulating the ATCA carrier 
environment specified in AMC.0. The 
virtual carrier provides the interconnect, 
power conversion, clock distribution and 
system management functionality needed 
to support up to 12 AMC modules. Some 
of this functionality may be implemented 
using components integrated as part of 


an active backplane. The most cost-effec¬ 
tive approach, however, is to implement 
this functionality using a dedicated vir¬ 
tual carrier management (VCM) module. 
Systems requiring high availability will 
likely deploy these modules in redundant 
pairs in order to eliminate the VCM as a 
single point of failure. 

The virtual carrier interconnect fab¬ 
ric provides the main connectivity among 
AMC modules in a MicroTCA enclosure. 
The VCM module acts as a dual-star hub, 
providing a central switch and high-speed 
lanes to each module. The half-duplex, se¬ 
rial lanes provide a scaleable bandwidth 
ranging from 3.125 Gbits/s to 12.5 Gbits/s 
per channel, compatible with the data rates 
supported by individual AMC modules. 

Versatile Packaging Option 

The MicroTCA specification suggests 
a number of packaging options, but does 
not define one as part of the spec. The 
suggested 19-inch rackmount MicroTCA 
chassis, for example, would range from 
2U to 6U and measure just 300 mm deep 
including cabling—a key requirement for 
many optical applications. The shelves 
would meet all applicable telecom equip¬ 
ment standards for central office and out¬ 
side plant applications, including NEBS, 
UL, OSHA, etc. 

MicroTCA chassis can accept any 
standard AMC module, including half- 
height/single-wide, half-height/double- 
wide, full-height/single-wide and full- 
height/double-wide modules. Figure 3 
shows a MicroTCA concept shelf. A typi¬ 
cal high-availability shelf would combine 
redundant VCMs and power modules with 
up to 12 AMC modules. The chassis would 
take power from an AC main or traditional 
-48 VDC telecom source, and convert it 
to 12V for delivery to individual modules. 
The power budget ranges from 20 watts 
for half-height/single-wide modules to 60 
watts for full-height/double-wide modules. 

To accommodate more irregular out¬ 
side plant, pole-mounted environments, 
work is underway to make the MicroTCA 
chassis available in a cube configuration 
(Figure 4), which measures eight inches 
wide by eight inches high by 200 mm 
(roughly eight inches) deep. Cubes can be 
used in a stand-alone mode. They can also 
be assembled into two-dimensional arrays 
and installed in a standard rack. 


200 mm 



Figure 4 


A cube version of MicroTCA 
can be used where smaller 
system requirements are 
appropriate. Cubes can also 
be assembled into larger 
system configurations. 


MicroTCA enclosures are not limited 
to standard shelves (such as ETSI or 19- 
inch racks) or cubes. These are popular 
options, but AMC’s compact size enables 
MicroTCA enclosures to be used in a va¬ 
riety of space-constrained applications 
where only a few modules (pico assem¬ 
blies) are needed to complete a system. 
Alternatively, MicroTCA enclosures can 
also be used to build large-capacity sys¬ 
tems with hundreds of AMC modules. 

The versatility of the AMC form-fac¬ 
tor makes it an ideal platform for building a 
wide range of scaleable, compact, low-cost 
MicroTCA systems for telecom, enterprise 
networking and customer premises appli¬ 
cations. This same versatility also makes 
AMC the preeminent mezzanine interface 
for building modular, high-availability 
ATCA systems. Together, ATCA, AMC and 
MicroTCA provide an end-to-end frame¬ 
work that addresses the full spectrum of 
high-availability networking applications, 
from core routers and WDMs, to converged 
customer premises equipment, d 
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PCI Express and Advanced Switching 


The Advanced Switching 

Advantage 


In the high-speed interconnect landscape, Advanced Switching Interconnect 
has several features that make it ideal for compute, storage or communica¬ 
tions system interconnect. This technology can be implemented at the blade 
or backplane level in applications ranging from small embedded systems to 
large compute platforms. 


by Chris Kendall 
StarGen 


T he candidates for high-speed in¬ 
terconnect include several differ¬ 
ent technologies, such as Advanced 
Switching Interconnect (ASI), Ethernet, 
InfiniBand, Serial Rapid I/O (sRIO) and 
PCI Express (PCIe). During the develop¬ 
ment of PCI Express, the technical con¬ 
tributors to this specification ensured its 
extensibility by layering the protocol for 
future advancements. Advanced Switch¬ 


ing is the first such extension. It shares 
the same physical and data link layers as 
PCI Express, which allows it to leverage 
the readily available and well understood 
IP that has been extensively verified and 
tested for interoperability (Figure 1). 

In addition, ASI maps seamlessly to 
any industry-defined PCI Express form- 
factor, such as Advanced TCA, AMC and 
ExpressModule. The additional capabili¬ 


ties of ASI are realized by enhancements 
to the transaction layer. These features 
include support for multiple hosts, true 
peer-to-peer communications, legacy 
compatibility, I/O virtualization, topology 
agnosticism, flexible protocol encapsula¬ 
tion, multicast transport, redundant con¬ 
nections and additional quality of service 
(QoS) and flow control mechanisms. By 
adding these capabilities, ASI expands 
the application space for PCI Express in 
compute, storage and communication sys¬ 
tems. 

Legacy Compatibility 

Advanced Switching utilizes a con¬ 
cept known as protocol interface (PI) for 
supporting different types of data move¬ 
ment within the interconnect. There are 
three types of Pis: services, native and 
tunneling, also known as encapsulation. 
The first encapsulation PI is PI-8, which 
facilitates tunneling of PCI Express traffic 
through an ASI fabric. Under this model, 
multiple virtual PCI Express hierarchies 
are formed within a common ASI fabric, 
each with a host CPU controlling one or 
more I/O devices and running legacy soft¬ 
ware. 


Define Services: 

• Transport 

• Services 
•Tunneling 


Perform Common Functions: 

• Initialization 

• Topology Discovery 

• Device Configuration 


Leverage Common PCIe ecosystem: \ 

• Sam compliant PCIe SerDes 

• Test & Measurement HW reuse 

• Familiary development environment 


PCI Driver Model 


ASI Driver Model 


Tunneling Pis 


PCI Plug & Play 
Model 


Services 

Pis 


Transport 
Pis 


ASI Fabric 
Management 


PCI Express 


Advanced Switching 


Same Link Characteristics 
CRC, 8B10B, Link Negotiation, Configuration 


Same SerDes 

Genl - 2.5Gb Gen2 - 5.0Gb 


Figure 1 


Advanced Switching Interconnect is based on a PCI Express foundation, 
and shares the same physical and data link layers. 
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PCI Express 
Hierarchy 1 


PCI Express 
Hierarchy 2 


PCI Express 
Hierarchy 1 



PCIe PCIe PCIe PCIe PCIe PCIe PCIe 

I/O I/O I/O I/O I/O I/O I/O 



Figure 2 


Multiple PCI Express hierarchies sharing a common ASI fabric. 


The PI-8 architecture allows balanc¬ 
ing of computational and I/O requirements 
and increases platform availability and 
serviceability. In addition, it allows the ex¬ 
pansion of PCI Express over extended dis¬ 
tances, typically up to 10 meters over cop¬ 
per, through ASI. Optical implementations 
have been shown to operate ASI x4 links 
over a distance of 100 meters. Another im¬ 
portant advantage is that the use of PI-8 
does not preclude the co-existence of ASI 
traffic over the same fabric (Figure 2). 

Flexible Data Movement 

Through the PI mechanism, ASI pro¬ 
vides orthogonal data movement models 
that are tailored for specific communica¬ 
tion policies. Advanced Switching Inter¬ 
connect is the only fabric that provides 
a simple load/store (SLS) model for data 
movement, i.e., PI-10. The SLS model is 
an adaptation of protocols that use a read/ 
write mechanism for data transfers, which 
does not require a copy of original data 
sent to an intermediate buffer previous to 
a transfer. 

In addition, PI-11 specifies the data 
movement PI for simple queuing (SQ), 
which provides a simple messaging model 
for packet-based communication us¬ 
ing familiar push/pull queue modeling. 
Socket data transfer, designated by P-9, is 
a mechanism that provides data synchro¬ 
nization and movement models for TCP 
socket-based connections. PI-2 represents 
the ASI model for generic data movement 
and includes segmentation-and reassem¬ 
bly (SAR) capabilities for packets larger 
than a specified ASI maximum packet 
size. Advanced Switching also supports 
the concept of software-based Pis. This 
allows software to construct packets that 
support other ASI protocol interfaces or a 
proprietary protocol interface. 

Trusted Peer-to-Peer 
Communications 

Advanced Switching allows several 
methods of peer-to-peer communication. 
One of these is through the mapping of lo¬ 
cal memory space in multiple computing 
domains using apertures, a component of 
the SLS model. Apertures are “windows” 


into memory from one peer entity to an¬ 
other, which are protected by various se¬ 
curity mechanisms. These mechanisms 
include path protection, access keys and 
sequence numbering to ensure a trusted 
path between totally monolithic or peered 
ASI endpoints. 

In the unlikely event of an error, ASI 
utilizes error detection and reporting far 
in advance of most fabrics available today. 
Event reporting is targeted for link cyclic 
redundancy check (CRC), packet CRC 
and invalid sequencing, among others. A 
true peer-to-peer relationship is facilitated 
in that there is no concept of an upstream 
or downstream bridge in the routing of 
ASI packets. There is only a notion of 
source and terminus. Peer-to-peer com¬ 


munication can also be achieved using the 
SQ model. This is useful for short mes¬ 
sages notifying CPUs of pending DMA 
transfers or other operations. 

Using data movement protocols like 
SQ and SLS, it is possible to open up an 
aperture into any other CPU’s local mem¬ 
ory and perform operations such as, but 
not limited to, placing a message in that 
CPU’s local queue indicating that data is 
ready to be transferred, then offloading 
that data transfer to a trusted path element 
in the fabric. This completely removes the 
target CPU’s interaction until the transfer 
is complete. In fact, the aperture mapping 
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In I/O virtualization, 
a single I/O endpoint 
has the intelligence 
necessary to service 
several CPUs by 
monitoring handles 
assigned to its usage 
from the different 
compute elements. 


mechanism included in native SLS allows 
any intelligent ASI endpoint to access any 
other ASI endpoint in a given fabric. This 
can be leveraged by utilizing ASI bridging 
on existing PCI Express-enabled devices, 
giving those devices the same ability as a 
native ASI endpoint. 

Protocol Agnosticism 

Advanced Switching’s protocol en¬ 
capsulation is not limited to PCI Express. 
A mixture of protocols can be simulta¬ 
neously tunneled through a single ASI 
fabric, making it a powerful and desir¬ 
able feature for next-generation modular 
applications such as multi-service rout¬ 
ers and embedded compute platforms. 
Advanced Switching’s path-based rout¬ 
ing architecture allows it to be protocol- 
agnostic, since switches only require an 
8-byte header to route packets. With this 
feature, applications currently using low 
performance proprietary protocols can be 
upgraded to ASI switching, and backward 
software compatibility can be maintained 
by tunneling of the existing protocols. 

Multicast Communication 

Not to be confused with the Ethernet 
broadcasting model, ASI supports true 
multicasting via multicast group IDs. This 
allows the combination of multiple end¬ 
points that are targeted to receive the same 
traffic into a single packet delivery at the 
source, rather than sequentially sending 
redundant packets to individual endpoints. 


Endpoints not considered members of a 
group can be excluded from receiving un¬ 
subscribed multicast group traffic, while 
unicast packets can still be sent to non- 
targeted endpoints concurrently with the 
multicast traffic. Any member of a given 
multicast group can be made a source 
or destination of the group. This feature 
becomes interesting in applications such 
as video and audio teleconferencing, or 
other applications where multiple end¬ 
points need to receive duplicate data from 
a common source. 

I/O Virtualization 

The concept of disaggregated I/O 
contrasts with the concept of I/O virtu¬ 
alization. Disaggregated I/O is accom¬ 
plished with the tunneling of PCI Express 
through a single ASI fabric. In this model, 
multiple PCI Express hierarchies share 
a common ASI interconnect, but within 
each hierarchy a single host-to-I/O rela¬ 
tionship remains. 

In I/O virtualization, a single I/O 
endpoint has the intelligence necessary 
to service several CPUs by monitoring 
handles assigned to its usage from the 
different compute elements. A single I/O 
device is given tasks by multiple CPUs 
and can operate on those tasks in the most 
efficient manner, as programmed. These 
individual tasks can be subsets of a larger 
operation, thereby allowing asynchronous 
parallel multitasking. Each task or sub¬ 
task is given a handle associated with a 
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target CPU, and the I/O device returns the 
results of a given task to the assigned tar¬ 
get. In this way, data flows can be main¬ 
tained at an optimum level since tasks can 
be performed asynchronously, resulting in 
a more efficient use of both the I/O device 
and the fabric. 

Advanced Switching provides for 
disaggregated I/O and virtualized I/O by 
allowing low latency, path-based routing 
from source to terminus. Additionally, the 
data movement models already discussed 
allow highly efficient CPU-to-CPU and 
CPU-to-I/O communication. The protocol 
agnosticism of ASI allows the simultane¬ 
ous transportation of differing I/O proto¬ 
cols through the fabric. Protection mech¬ 
anisms of ASI allow data to flow from 
endpoint to endpoint, and ensure that it 
is only delivered to the correct terminus 
point in the fabric. These factors lead to a 
design where I/O can be completely dis¬ 
aggregated from the compute elements of 
a given system (Figure 3). 

Robust QoS 

Advanced Switching offers extensive 
QoS through various components, in¬ 
cluding different service levels for traffic 
flows and congestion management. Traffic 
classes are logical entities that are mapped 
to physical virtual channels (VCs). Up to 
16 unicast and 4 multicast VCs are speci¬ 
fied in ASI. Credit-based flow control de¬ 
livers a link-level-like flow control mech¬ 
anism between two interconnected ASI 
devices. This flow control is available for 
each individual VC of a port, resulting in 
effective QoS between traffic types. 

Status-based flow control, unique 
to ASI, is a more granular flow control 
mechanism that is relative to the VC of 
the particular segment and also to the VC 
of the next destination segment. It is initi¬ 
ated at the egress or output of the interface 
back toward the previous segment. This 
allows a more sophisticated mechanism 
for minimizing head-of-line blocking for 
one-, two- or three-stage fabric topologies. 
Injection rate limiting provides connection 
queues for flow isolation at a packet source 
enabling fine-grained rate adaptation. 

Scalability 

Advanced Switching fabrics allow 
hot-swapping and “hot-joining” end¬ 
points. The endpoint can be a single 



Figure 4 


Board-level ASI products include the Kontron AT8901 AdvancedTCA 
Base and Fabric Hub Board, which supports both Gigabit Ethernet and 
ASI switch fabric options. Built with the Merlin ASI Switch module from 
StarGen, the board provides ASI connectivity to the fabric interface. 


I/O or compute device, or a completely 
monolithic system consisting of a root 
complex, memory and I/O, running un¬ 
der an entirely different operating sys¬ 
tem than the system to which it is being 
connected. This is achieved by adding 
a “DL_PROTECTED” state to the data 
link (DL) engine just above the physical 
layer. If an endpoint is hot-swapped or 
hot-joined, the DL layer detects this as 
a fabric event, and places the link in a 
protected state. In this state the physical 
layers connect and exchange link infor¬ 
mation. Once the link is up and verified, 
ASI message events, designated by PI-5, 
are allowed through the link in order to 
let a fabric manager verify the new end¬ 
point and resources. Once the new end¬ 
point is verified, the DL returns from a 
protected state to an active state. 

If the new endpoint is a complete 
system also running a fabric manager, the 
implementer has multiple options for how 
to proceed. One possibility is that the two 
systems can act as independent, mono¬ 
lithic entities that share the same fabric. 
In this case, they merely share resources 
based on the data movement apertures 
outlined in the SLS protocol by their re¬ 
spective fabric managers. Another possi¬ 
bility is that one system can take “owner¬ 
ship” of another. This is accomplished by 
initiating a spanning tree election in order 
to determine the primary fabric manager. 
Once this is determined, data is exchanged 
to allow proper data movement through 
the fabric to its new set of endpoints. 

Thus, using the hot-swap and hot-join 
features of ASI allows a scalable fabric that 


can be designed to facilitate the system ex¬ 
pansion, monitoring and flexibility needed 
in complex compute environments. 

Redundancy 

Redundancy in a fabric interconnect 
is envisioned using duplicated topologies. 
Advanced Switching allows active-passive 
and active-active designs using star, dual¬ 
star, meshes and completely unique imple¬ 
mentations of fabric topologies. The use 
of primary and secondary fabric manag¬ 
ers, as well as ASI event messaging, allow 
failover mechanisms to be implemented 
covering minor to catastrophic fabric 
events. Data movement models for per¬ 
forming dual casting of packets through 
separate fabrics to the same terminus are 
available. Additionally, SLS apertures can 
be configured to allow primary and sec¬ 
ondary paths through a given fabric, or its 
redundant partner for packets to route to 
their target endpoints. This makes ASI an 
excellent candidate where redundancy in 
a system is required. 

Advanced Switching Interconnect 
technology can be implemented at the 
blade or backplane level in compute, stor¬ 
age or communications systems (Figure 
4). The benefits of ASI are many for ap¬ 
plications ranging from small embedded 
systems to large compute platforms, d 
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Executive 

Interview 


RTC Interviews 
Diamond Systems’ 
Jonathan Miller 



RTC: Sales of PC/104 and its deriva¬ 
tives appear to be skyrocketing and 
these products are rapidly displacing 
the other standards-hased products in 
a variety of applications. In our annual 
market report last year, you believed 
our estimate for PC/104 and derivative 
products was low. Can you give us your 
estimate of what the worldwide volume 
of business in PC/104 is? Can you pro¬ 
vide some insight into where you expect 
it to go in the next couple of years? 

Miller: If you consider just the PC/104, 
PC/104+, PCI-104 CPU and add-on module 
market, the worldwide volume is probably 
somewhere around $100 million. But if you 
include all the CPUs that have PC/104 ex¬ 
pansion sockets on them, the market is likely 
2-3 times that size. In the future we will see a 
steady increase in the “PC/104-expandable” 
market, meaning small form-factor CPUs 
with PC/104 expansion sockets on them. 
That will naturally improve the prospects 
for PC/104 expansion modules as well. 

RTC: The small form-factor and inher¬ 
ent ruggedness of PC/104 have made it a 
favorite in a variety of applications from 
the military to retail POS installations. 
As derivatives of the original form-fac¬ 
tor evolve, such as EPIC, EPIC Express 
and others, has there been the same ef¬ 
fort to provide the same robust mechan¬ 
ics as in earlier versions? What has been 
done to ensure the ruggedness and as¬ 
sure our readers of that? 
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Miller: The concept behind PC/104 is a 
combination of small size, ease of expan¬ 
sion, multiple vendors and ruggedness. The 
concept behind form-factors such as EPIC 
and Biscuit is primarily the combination of 
small size and low cost. Ruggedness and 
even expandability are second in line. Hav¬ 
ing PC/104 expandability doesn’t necessar¬ 
ily mean that these form-factors inherit all 
the attributes of PC/104. 

First of all, it’s pretty clear that the vari¬ 
ous PC/104-expandable form-factors have not 
been designed with the same goals as PC/104 
itself. Specifically, the use of consumer-type 
I/O connectors on many EPIC and Biscuit 
CPU boards identifies these products as 
limited to low-cost, office-environment ap¬ 
plications. You don’t see RJ-45 and Mini- 
DIN connectors in military applications. In 
addition, rugged applications generally shun 
snap-in memory modules that are prone to 
popping out in a high-vibration environment. 
Ruggedness is like quality—it permeates the 
entire product, and the weakest link decides 
the overall capability. In order to assess a 
product’s ruggedness, you have to look at 
the choice of components, connectors, power 
and cooling requirements, memory and mass 
storage options and so on. And then of course 
there’s the whole packaging issue. 

Secondly, as processor computing 
speeds and power requirements go up, the 
market has split into two distinct segments: 
rugged and performance. A Pentium M 1.6 
GHz CPU with a large heat sink and fan, 
coupled with a 168-pin memory module 
mounted vertically and rated at O-6O0C, 
does not meet my definition of rugged. But 
there is a whole range of customers that 
want that type of board. So ruggedness is 
not the only concern for embedded CPU 
developers. Each project has its own re¬ 
quirements, and the designer’s responsibil¬ 


ity is to prioritize those requirements and 
select components accordingly. 

RTC: COM Express has been getting its 
share of publicity recently. And, while it 
represents a slightly different approach 
to open modular systems, it is still judged 
by some as competitive with approaches 
such as PC/104 and PC/104+. What 
is your view on how the two compare? 
What are the strengths and weaknesses 
of each approach? 

Miller: COM Express belongs to a differ¬ 
ent category of embedded CPU board than 
PC/104, and it therefore addresses a differ¬ 
ent market. We rarely encounter a PC/104 
customer who is also considering a COM 
module. COM, or Computer-on-Module, 
means a CPU board that is essentially a 
large component that is mounted on a larger 
custom baseboard. COM boards, like ETX 
and COM Express, are targeted toward 
large volume customers who have in-house 
engineering teams or budgets to support the 
development of that custom baseboard. 

PC/104, on the other hand, is base- 
board-free. You can build a PC/104-based 
system without knowing anything at all 
about board-level design. This is why 
PC/104 has become so popular—it provides 
computing power to the masses, and it dra¬ 
matically lowers the cost of entry to a com¬ 
puter-enabled product design. So I think that 
PC/104 and COM can live side by side, each 
addressing its particular market very well. 

RTC: PC/104 and PC/104+ have been 
dramatically successful because they 
fit into such a wide range of applica¬ 
tions from medical instrumentation to 
transportation systems (subways, trains, 
etc.). There is also some indication that 
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PC/104 and derivatives are starting to 
be used in a wide variety of other appli¬ 
cations such as security from biometric 
recognition systems to document read¬ 
ers for passports and driver’s licenses. 
What application areas do you believe 
will help spur the growth of PC/104 over 
the next several years? 

Miller: There are still tremendous growth 
opportunities for PC/104. The increase in 
the number of form-factors with PC/104 
expansion opens up more sockets where 
PC/104 boards can find a home. Also I 
think it’s important to expand our focus 
beyond PC/104 to the small form-factor 
embedded computer in general. It’s actu¬ 
ally quite possible that in the next few years 
we may see a reduction in the PC/104 CPU 
market, due to several factors. The new 
growth applications (video capture, image 
analysis, communications) require ever 
higher data rates or computing power that 
call for ever higher levels of CPU perfor¬ 
mance, yet the processor chipsets that can 
deliver that performance can’t fit easily, if 
at all, on a PC/104 board. So these appli¬ 
cations will migrate to other form-factors. 
But the total market for small form-factor 
computing will grow significantly. 

A few dynamics will affect the pure 
PC/104 add-on market. First of all, Dia¬ 
mond Systems’ goal is to integrate as much 
I/O as possible onto the CPU board, in or¬ 
der to reduce the need for add-on PC/104 
boards. Other vendors are starting to do 
the same. I also think that as CPU prices 
continue to come down, there will be down¬ 
ward pressure on I/O boards as well, so the 
total value of the pure PC/104 market won’t 
grow as fast as the number of boards. 

RTC: While many PC/104 companies 
such as Diamond Systems are relatively 
large, there seem to be relatively few 
large publicly held entities specializing 
in PC/104. Do you agree? If so, is this 
because relatively smaller firms are bet¬ 
ter equipped to handle the diverse needs 
of the customer base? Is this in part be¬ 
cause the industry is still maturing? 

Miller: The latest market research I’ve seen 
indicates that PC/104 is only about 7% of 
the global board-level embedded comput¬ 
ing market. Large companies go after large 
markets, while startup companies go after 


niche markets, where the barrier to entry 
is lower due to easier access to technol¬ 
ogy, lower product development costs and 
less competition from 800-pound gorillas. 
PC/104 fits this category well. Many PC/104 
companies are small enough that a potential 
market of a few hundred boards is enough 
to justify a new product design. But a larger 
company would never devote its resources 
to such a small market. Also, the largest em¬ 
bedded computing companies are all build¬ 
ing systems, while the PC/104 industry is 
just waking up to that possibility. 

RTC: Looking at the previous question, 
the open-standards industry has been 
going through unprecedented consolida¬ 
tion. Companies such as Curtiss-Wright, 
Mercury Computers and SBS alone have 
been responsible for over a dozen acquisi¬ 
tions in recent times and there are likely 
to be more coming. However, notably, to 
our knowledge, none of those acquired 
has been in the PC/104 business despite 
the fact that PC/104 has been active in all 
the same application areas these compa¬ 
nies specialize in. Further, with a single 
possible exception, there has been virtu¬ 
ally no consolidation among the more 
than 50 makers of PC/104. To what do 
you attribute this lack of consolidation 
frenzy in an industry that is fraught with 
it? Is this likely to change? Why? Why 
have these larger consolidated companies 
eschewed PC/104 and derivatives? 

Miller: It’s not just a question of whether 
there is a buyer, but also whether there are 
willing sellers. In fact there was a wave of 
consolidation a few years ago, with the big 
fish eating the little fish, and then an even 
bigger fish coming along and eating the big 
fish! Then more recently we saw the cross¬ 
continent merger of Eurotech and Parvus. 
I estimate that in the next 5 years you will 
see a continuing shakeout as the Taiwanese 
manufacturers continue to exert extreme 
pressure in both price and time-to-market. 

PC/104 is just one segment of the over¬ 
all embedded computing industry. In the 
larger landscape, mergers will continue 
to happen, and PC/104-centric companies 
will experience their share of action, if their 
owners are open to the concept. 

One distinct likelihood is the acquisition 
of some American companies by the larger 
Taiwanese manufacturers as they attempt to 


acquire a stronger foothold in the U.S., which 
currently represents about 50 percent of the 
worldwide market for embedded computers. 

RTC: PC/104 and PC/104+ have man¬ 
aged to keep power budgets down while 
increasing performance for many gen¬ 
erations through the use of low-power, 
slower processors and clever design. 
However, as products move to such stan¬ 
dards as EPIC and EPIC Express, is it 
not likely that higher I/O bandwidth will 
call for higher performance processors 
and their attendant higher power bud¬ 
gets? How do manufacturers of these 
systems and subsystems plan to deal with 
the higher power and thermal budgets? 

Miller: Well, if you base your designs on 
some of the more popular American ven¬ 
dors’ chips, you can pretty much forget 
about low power, at least from the per¬ 
spective of embedded systems. But if you 
look to alternative chip vendors, there are 
still options that provide sufficient com¬ 
puting power for emerging applications, 
while keeping within reason both cost and 
power consumption. For example, Dia¬ 
mond Systems has just released the new 
Elektra, a PC/104 CPU with a 200 MHz 
Pentium processor that consumes only 0.5 
watt more than our previous 100 MHz 486 
product. And VIA has just announced a 
1 GHz processor that consumes only 3.5 
watts. This raises the possibility of an ex¬ 
tended temperature, fanless CPU with 1 
GHz performance, something that is more 
or less unavailable today. 

Also, remember that the market is quite 
fragmented: Many customers don’t need 
low power or extended temperature, they 
just want performance. System developers 
are always involved in a juggling act with 
their priorities, so the choice between high 
performance and low power is nothing new. 
In my view, a much more important issue 
on the horizon now is the choice between 
low cost and long-term availability. 

RTC: Despite there being somewhere in 
the area of 50+ vendors of PC/104 and 
related products, virtually all appear 
to be prospering. Is there in fact direct 
price competition? What is it that al¬ 
lows this many competitors to survive 
and prosper? Is it that there is so much 
business out there that competition isn’t 
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felt? Does this mean that we can expect 
a continued dramatic expansion of this 
part of the industry? 

Miller: There is extreme price competition 
in the CPU market. Again, this is largely a 
result of the strong performance of the Tai¬ 
wanese vendors, which is resulting in many 
customers re-evaluating whether they are 
willing to pay the price for long-term avail¬ 
ability and rugged performance that is 
typical of American suppliers. For certain 
applications (military, medical, transpor¬ 
tation) there is no question that both rug¬ 
gedness and long product life are worth the 
extra cost. But for other applications, that 
may not be the case. Price pressure is there¬ 
fore largely due to the requirements of the 
applications that represent today’s growth 
opportunities. Also, the continuing global¬ 
ization of the world’s markets means that 
every vendor is competing with every other 
vendor, adding to the number of customer 
choices and driving down cost. Hey, maybe 
I should apply for Greenspan’s job... 

RTC: Our readers are always looking to 
see what’s in the future and how they are 
going to address their future designs. 
And, new technology is exciting. Can you 
provide our readers with a little peek 
into the future of what they can expect 
from Diamond Systems? 

Miller: Look for continuing efforts in the 
area of integration, both board-level and 
system-level. The time-tested formula for 
success is miniaturization and cost reduc¬ 
tion. One way to get there is through inte¬ 
gration. As I mentioned earlier, our strat¬ 
egy, which we call “2 in 1,” is to build the 
I/O right onto the CPU, so that a system 
designer needs fewer boards. Fewer boards 
mean lower cost, less assembly time, a 
smaller enclosure, less repair time, less 
purchasing and inventory overhead and so 
on. The customer benefits of such a strat¬ 
egy are quite compelling. 

RTC: There’s been reported a growing 
movement toward OEMs looking to pur¬ 
chase completed systems and subsystems 
from their embedded computer vendors 
rather than just boards. Do you find this 
to be the case? On the other hand, the va¬ 
riety of available modules allows an OEM 
to mix and match to meet his application 


demands, even selecting modules from 
different vendors, without doing a lot of 
detailed engineering. How do you see the 
mix of these trends in the PC/104 space? 
Miller: There’s no one solution to fit all 
needs. Many customers want to outsource 
as much as possible, leading to an increase 
in the opportunities for companies that 
want to get involved in custom design or 
system integration. However, other com¬ 
panies want to maintain greater control 
over their technology base, especially 


if they see it as critical to their business 
strategy. For both customer types, PC/104 
and other small form factor CPUs provide 
a great solution. The system integrator, as 
well as the in-house designer, can select 
from a wide range of options to build a 
system just right for their needs. In addi¬ 
tion, the open standard and wide supplier 
base enable them to extend the life cycle 
of any particular design, by allowing for 
technology refreshes at very low cost, d 
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ZX6000 24 port Gigabit Ethernet 
PICMG 3.0 Ethernet Switch 



ZX7000 48 port Gigabit Ethernet 
PICMG 3.0 / 3.1 Ethernet Switch 
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Integrated, Pre-tested Solutions 

Complex systems can be built out of a few basic building 
blocks.The Modular Switching Solutions for ATCA from ZNYX 
Networks follow the same idea. Relatively simple modules 
may be used for basic architectures, starting with systems 
that use only the ATCA Base Interface.Then the same mod¬ 
ules are used with others to make more complex ATCA solu¬ 
tions, involving PICMG 3.1 data planes from single Gigabit 
links to multiple links, and from there to full 10Gigabit 
Ethernet backplanes. 

Versatility is key. 

Modularity is key. 

Only ZNYX Networks' Modular approach can span the ATCA 
universe from the very simple to the very complex with one 
product line, using the ZX6000 as the common base plat¬ 
form. We support fully integrated fabrics that have non- 
blocking switching service between Base and Fabric and also 
total separation between Control and Data planes, complete 
with independent control processors. 

The true Linux environment of OpenArchitect® makes mod¬ 
ular innovation possible. Whether the need is for the latest 
protocol, a new VLAN setup, or a new peripheral, 
OpenArchitect® makes what Linux has to offer available. 

Contact sales@znyx.com for more details. 
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Build a Better Distributed 
System with a Hybrid 
Linux-RTOS Approach 

For many kinds of systems, the best approach is not a decision between 
Linux or an RTOS, but rather a combined Linux-RTOS design. Such a 
hybrid approach entails meeting a number of requirements to be successful. 

by Michael Christofferson 
Enea Embedded Technology 


T here is no question that Linux has made great strides in 
recent years regarding its suitability for real time and high 
availability—witness the Linux 2.6 kernel and carrier-grade 
initiatives. It is suited to server applications, network/system 
management applications, so-called control plane applications 
and so on. Notwithstanding its open source aspects, which are 
another topic, the primary value proposition of Linux for many is 
the universe of applications that it supports. The fact that Linux 
now supports many real-time characteristics allows this value to 
be leveraged in embedded systems. 

However, in the realm of complex distributed systems this 
value proposition doesn’t always apply to all nodes or functions of 
the system, although it certainly applies to some. Often there are 
parts of the system that require very specialized functionality and 
that need to be highly tuned for performance, resource consumption 
and sometimes footprint. Often, hardware architectures reflect this 
by featuring a combination of general-purpose CPUs and more spe¬ 
cialized CPUs, microcontrollers or DSPs. This is characteristic of 
most communications systems, including many wireless terminals 
or mobile handset devices. More generally, this is characteristic of 
systems that are “traffic bearing,” i.e., they feature high throughput 
requirements as opposed to more control type systems wherein re¬ 
sponse time to discrete events is the primary focus of the system. 

Traditional RTOSs that have been developed specifically 
to address performance and footprint, and that have often been 
“tuned” for specific processors, are the ideal choice for some 
of the nodes/functions of such systems. Choosing a single OS 
for development of any system always helps lower development 
or NRE costs. But OEMs building such traffic bearing systems 
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most often win by producing the best price/performance charac¬ 
teristics of the delivered product. 

A design utilizing both general-purpose and more special¬ 
ized processor components along with a hybrid Linux-RTOS op¬ 
erating environment often proves the best choice for achieving the 
winning edge, and such hybrid approaches actually dominate the 
market today. As one OEM representative put it, “Let Linux do 
what Linux does best, and let RTOSs do what RTOSs do best.” 

Particularly suited for a hybrid OS approach are the several 
kinds of traffic bearing systems or even devices. These include 
telecom/networking infrastructure, particularly access, edge and 
core devices as well as SOHO systems along with systems in other 
markets, like automotive telematics, some Mil-Aero systems and so 
on. They include multicore devices and wireless terminal devices. 

Basic Requirements 

There are two primary requirements for linking Linux and 
RTOSs. They must use a common inter-process communication 
(IPC) model and there must be a degree of code portability at 
some level. The IPC model must feature some robustness in sup¬ 
port of advanced fault management and recovery, as so many 
complex distributed systems feature such requirements. For a hy- 
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Multi-blade HA-capable Netcom System. 
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When you're developing your next product, you need more than just a board: you need a solution. 
That's why we offer these important services to enhance our embedded computer boards and help 
customers like you get to market more quickly and with lower costs. 
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CUSTOMIZATION 

Diamond System's customization program 
offers a proven and reliable approach for 
satisfying requirements that can't be met 
with off the shelf products. We offer mod¬ 
ifications to standard products, as well as 
full-custom designs. 
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Customized version of Diamond 
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of the services we offer to ensure that you get the quality and reliabil¬ 
ity you need. 


SOFTWARE 
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brid Linux/RTOS system, a direct message-passing model works 
well if it is properly implemented to support delivery transpar¬ 
ency, redundancy as well as failure and failover management. 

Asynchronous message passing is the preferred technique 
for communication between tasks on multiple processors or on 
multiple boards since it offers higher performance and is safer 
and more reliable than other approaches. This is why the latest 
switched fabric technologies (RapidIO and AS/PCI Express) have 
message-based CPU interconnects. Message passing is conceptu¬ 
ally simple and intuitive. The use of asynchronous message pass¬ 
ing for inter-task/process communication prevents the propaga¬ 
tion of faults. Message-passing also works exceptionally well with 
hardware memory protection between subdivisions, ensuring that 
faults in one subdivision cannot corrupt other subdivisions. 

Semaphores, mutexes, event flags, Unix-style signals and 
other traditional operating system mechanisms for inter-task/ 
process communication are not suitable for distributed embed¬ 
ded systems. They also do not fit well with memory protection 
schemes. Ideally, the same asynchronous message-passing model 
should be used for both communications between tasks within 
the same processor, and also for communications between tasks/ 
processes that will reside on different processors. 

Since each node may have its own operating system and 
perhaps differing interconnects, the message-passing mecha¬ 
nism should support both reliable and unreliable communication 
links. In order to support the above “robustness” in a distributed 
system, the IPC model should be capable of certain features. It 
should support and dynamically identify remote nodes, processes 
and tasks, applications and communication links; monitor them and 
provide prompt notifications to the system if any of these fail. It 
should support redundant nodes and communication links. There 
should be transparency of location between communicating entities, 
both intra-CPU and inter-CPU. This then provides the basis for some 
sort of portability at the application level. And it should be able to 
use any type of physical interconnect and adapt to any protocol, thus 
providing a single software model for inter-CPU communications. 

The key to linking Linux and an RTOS in this model is to 
provide a common set of APIs that makes all processes and tasks 
in the system OS-transparent. This will enable Linux applications 
to seamlessly communicate with CPUs in the system running an 
RTOS. However, one drawback here is that there are currently no 
real standards for message-passing IPC models in the embedded 
industry. Still, there are some offerings available, and there are 
currently several initiatives that aim to provide some standards, 
and as importantly, in an open source context. 

In reality, absolute code portability—the ability to physically 
drop code developed from Linux to an RTOS unchanged—for such 
hybrid systems is overrated. The notion that any application may be 
moved to another node in the system is intellectually powerful, but 
it is rarely done in practice. Changes to characteristics and behavior, 
not to mention the hardware environment, often override such at¬ 
tempts once an application is developed. Most often, the designers 
of the system created the hardware architecture to reflect a particular 
portioning of functionality for a reason. Once such a partitioning is 
made, it rarely changes except in minor ways during development. 

Later, if a “next generation” of the system is developed, the 
hardware and software architecture changes often obviate direct 


porting of applications to the new environment. So insisting on ab¬ 
solute code portability between Linux and the chosen RTOS(s) is not 
really required. However, a “softer” level of portability that is useful 
can be achieved by using the design discipline that message pass¬ 
ing be used between all applications/processes/tasks in the system. 
This is a cleaner model, it’s high performance and is distributed if 
location transparency is supported. Maintaining interfaces to other 
applications and system services is the biggest issue in porting. 


Systems Best Suited to a Linux-RTOS Approach 

Traditional “big iron” telecom/datacorn systems are ideally 
suited to a hybrid OS design, especially in the access, edge and 
core domains. SOHO systems are now also increasing in com¬ 
plexity and specialization, and could be candidates. The same 
applies to systems in other markets, like Automotive Telematics, 
some Mil-Aero systems and so on. Ligure 1 illustrates the basic 
outline of these types of systems where the hardware reflects the 
different requirements placed on the operating systems. 

In a hybrid design, Linux would run on the management/con¬ 
trol cards, performing such functions as control plane management 
of system, network management (or interface to external network 
management systems), system provisioning or configuration and 
system monitoring and maintenance. Linux offers strong capabili¬ 
ties here. Line cards, which may have other CPUs, microcontrollers 
and DSPs in any combination would be under the RTOS. These 
would be responsible for all data plane activities—actual traffic 
or traffic management/control and so on. RTOSs provide the best 
fit for many if not most scenarios here, especially if the line cards 
are very high-performance and specialized processors like DSPs, 
and also especially if these cards have high-performance or tight 
deadline requirements. Again, having a common robust, high-per¬ 
formance IPC model makes such a system feasible. 

The market is currently seeing a strong trend toward multi¬ 
core CPUs. Prom the hardware perspective, a multi-core approach, 
rather than higher clock speeds, is the best way to improve per¬ 
formance within reasonable power and footprint constraints. The 
“hardware-oriented” view of such a device is that a formerly single 
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LH = Load Handling 
Multicore Hybrid OS System—with load balancing. 
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Figure 3 


Hybrid Linux—RTOS with Linux “Virtualization.” 


CPU software design can run with better performance by employ¬ 
ing Symmetric Multi-Processing (SMP) techniques. Indeed, SMP 
was originally designed to boost the performance of a single appli¬ 
cation across multiple CPUs, without requiring a design change. 

However, multi-core doesn’t necessarily mean simply im¬ 
proving performance of a formerly single CPU design. It can 
also offer an opportunity to treat the device as a mini-distributed 
system in which each core has dedicated or partitioned function¬ 
ality. In fact, the vendors of such devices are making specific 
allowances to support such partitioning with, for example, sepa¬ 
rate caches for each core, more flexibility in shared and separated 
device management and in the design of interrupt controllers. 

This mini-distributed approach is sometimes referred to as 
asymmetric multi-processing (AMP). The AMP concept then 
leads directly to the notion of a hybrid OS approach, with the 
model being the same as for a more traditional distributed telecom 
system—Linux on one core as the control plane/management 
node, and an RTOS on the other core(s) being the high-speed 
“data pump.” Again, traffic bearing systems are the most likely 
candidates for such a design approach—and the value proposi¬ 
tion is lower chip count/real estate and higher price/performance 
profiles. Many OEMs are currently using this model. 

Additionally, there is much work being done on a modified 
AMP model, in which some level of load balancing of certain 
functions may be achieved without resorting to SMP where all 
functions are load balanced or dynamically distributed at invo¬ 
cation. Any load balancing requires the same OS to be execut¬ 
ing on the cores. Nonetheless, with the proper allowances for 
shared hardware control, and common memory management, 
this “load balancing” AMP solution under an RTOS may coexist 
with Linux as well. This is an ideal scenario, because the traf¬ 
fic bearing cores running an RTOS would most benefit from a 
load-balancing scheme. So it is still possible to achieve the best 
of both worlds—a load balancing RTOS for highest performance, 
and Linux for control and management (Figure 2). 

In Figure 2, if a common device management scheme were im¬ 
plemented, then Linux would run on one node, and an RTOS would 
run on the other two nodes with load handling, represented by applica¬ 
tions being instantiated and distributed across cores. This represents 
true asymmetric behavior, but with performance optimization. Com¬ 
mon device management includes common memory management or 
mapping as well as common device control and configuration, so that 


Linux and the RTOS do not collide on device resources. 

Wireless terminals are another appropriate platform. Mobile 
phones, for example, are already multi-processor distributed sys¬ 
tems, and these usually feature a hybrid OS approach (not neces¬ 
sarily Linux). In this market, more functionality and more perfor¬ 
mance (now that video and downloadable services are coming) at 
extremely low costs are the big drivers, and the current trend is to 
move toward single-chip solutions. This presents an interesting 
application of a hybrid OS approach—Linux and an RTOS on 
the same device, where Linux would run the displays and control 
the features, but the RTOS would control the radio communica¬ 
tion (baseband RF) protocols. In effect, one may still view such 
a situation as an example of a distributed system. The original 
distributed system characteristics, as reflected in different OSs 
on different chips, are simply being mimicked on a single chip 
with the same OSs. In this case one of them is Linux! 

Many approaches to a combined Linux-RTOS solution have 
been proposed and implemented. The most promising one for this 
particular problem is a “virtualization” approach, wherein Linux 
runs as a task under the RTOS, which controls the basic CPU 
(interrupts and device management, memory management and 
scheduler), but it does so using a virtual machine implemented in 
the RTOS that emulates all system calls (Figure 3). 

In this fashion, pure native Linux is employed without modi¬ 
fication. This model offers complete separation of Linux and the 
RTOS and is clean in the sense that it requires no Linux modifica¬ 
tions. The penalty is in the performance of Linux due to the virtual 
machine. But then the performance requirements on Linux are low 
in this case. The virtualization approach is well proven and reliable 
as witnessed by the many Linux virtual machine implementations 
on Windows, for example. The value proposition is that Linux 
may be used for graphics, display, control, standard applications 
and standard device management, but the RTOS manages all the 
time-critical functions involving communications and streaming 
audio/video from embedded flash and such. So we have the same 
story—another case where Linux and an RTOS may happily coex¬ 
ist and offer a better technical solution than either could alone, d 

Enea Embedded Technology 
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Java and Linux Make 
Natural Partners for 
Soft Real-Time Solutions 


For large, complex systems with less than totally critical timing 
requirements, the combination of Linux at the OS level and Java at the 
application level can result in systems that are flexible and configurable 
while offering performance that can meet soft real-time requirements. 


by Kelvin Nilsen 
Aonix 


M odern soft real-time systems tend to be much larger, 
more dynamic and more complex than previous gen¬ 
erations of products. They typically include multiple 
layers of software, some of which incorporate networking 
infrastructures that must be securely networked with other 
computers as part of the greater Internet. To address such 
challenges, developers are looking to larger, more complete 
development solutions where the tools themselves are part of a 
universal framework. 

Linux as an open source OS solution and Java as a ubiqui¬ 
tous language have come to play such a role. Among the benefits 
of Linux are its support for all popular embedded CPU architec¬ 
tures and single board computers, self-hosted development and 
an abundance of ready-to-run open-source software components. 
Java builds upon these benefits by enabling developers to quickly 
and easily develop powerful new capabilities as custom software, 
and allowing them to integrate into their custom software solu¬ 
tions a huge variety of off-the-shelf Java software components. 

There are a number of general challenges associated with 
soft real-time development that differ from those normally as¬ 
sociated with real-time development. Soft real-time systems tend 
to be large, dynamic and complex. Because of their size and 
complexity, it is not practical to use the rigorous analysis dis¬ 
ciplines for development of this software that are typically used 
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for hard real-time development. Rather than determine resource 
needs through meticulous analysis, the soft real-time developer 
typically determines resource needs empirically, measuring 
how components behave. Developers use statistical and heuristic 
techniques to increase the probability that deadlines will be met 
and to maximize the utility provided by the soft real-time sys¬ 
tem whenever it is not possible to satisfy all deadlines because of 
work overloads. 
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The quip, “Hard real-time is hard. Soft real-time is harder,” 
accurately indicates the increased difficulty in building complex 
soft real-time systems. In general, developers choose to use soft 
real-time techniques whenever they are not able to statically 
guarantee availability of all the resources required for reliable 
compliance with hard real-time deadlines. 

This usually results from a combination of several possible 
factors. For one thing, it is not always possible to determine an 
absolute upper bound on the frequency or magnitude of workload 
processing requests, and the software that processes the work¬ 
load has unpredictable execution times. For this class of system, 
it would be too costly to guarantee availability of all the resources 
required to ensure hard real-time operation in the worst-case sce¬ 
narios. In addition, there could be conflict for resources because 
the desire to maximize the utility of available computation re¬ 
sources often requires utilization of resources that cannot always 
be guaranteed available at a given time. 


language for complex, dynamic real-time systems, almost all ex¬ 
isting deployments exploit a combination of Java code to imple¬ 
ment the more dynamic and highest levels of the system hierar¬ 
chy, along with C or C++ code to implement the lower layers of 
the system hierarchy. These lower layers are often characterized 
by more static behavior and more demanding performance and/ 
or memory footprint requirements. 

Among the other reasons Linux is increasingly used in 
mission-critical systems are the various improvements intro¬ 
duced with the newly released Linux 2.6 kernel. This adds 
significant enhancements to support improved scalability, 
reliability and security. Among these features, there is im¬ 
proved support for multiprocessor systems, larger memory 
configurations and better real-time predictability with tighter 
real-time responsiveness. 

As a programming language supported on top of Linux, Java 
is an appropriate platform for development of large and complex 


Linux and Java are already being used together in a variety of mission-critical soft 
real-time systems. However, many of these have not yet authorized public disclosure 

of their activities. 


Because of their size and complexity, and the frequent need 
for dynamic reconfiguration of the processing workload, soft 
real-time systems often need much more underlying support in¬ 
frastructure than is typically provided by commercial real-time 
operating systems. Traditionally, mission-critical soft real-time 
systems have most commonly run on commercial variants of 
Unix, such as AIX, HP-UX and Solaris. During recent years, 
Linux has matured significantly. It has now demonstrated its 
value in many high-reliability commercial deployments. With 
this proven reliability, Linux is increasingly deployed in mission- 
critical applications that were previously dominated by commer¬ 
cial Unix operating systems. 

Because Linux is supported by a very large community 
of open-source developers, Linux configurations are available 
to run on every popular microprocessor CPU architecture and 
board. For this same reason, a wealth of device drivers is avail¬ 
able to support all sorts of different communication protocols 
and I/O devices, including USB and Firewire (IEEE 1394). More 
recent initiatives, such as the carrier-grade Linux effort, establish 
standard mechanisms for implementation of high-availability 
systems, including support for warm-standby operation and ap¬ 
plication failover, as well as support for hot-swappable devices. 
Real-time virtual machines running on top of Linux provide 
j a v a - i o library access to all of the devices supported by the 
Linux configuration. 

As a POSIX-compatible operating system, Linux also offers 
a stable platform on which to deploy legacy software components 
written in C or C++. Many open-source legacy software compo¬ 
nents are freely available to be incorporated into Linux-based 
system designs. Even though Java makes an ideal programming 


dynamic systems. Its high-level support for portable and scal¬ 
able object-oriented development supports easy development, 
integration and maintenance of large systems comprised of inde¬ 
pendently developed software components. Most programmers 
find that they are approximately twice as productive developing 
Java code than C++ code, and up to ten times more productive 
during the integration and maintenance of embedded real-time 
Java software. 

Historically, doing real-time software with Java has been 
difficult because traditional Java virtual machines experience oc¬ 
casional long execution pauses while they perform automatic gar¬ 
bage collection. Depending on the memory configuration, these 
pauses may last up to thirty seconds at a time. Real-time virtual 
machines allow soft real-time execution of standard-edition Java 
components, supporting reliable compliance with timing con¬ 
straints ranging from 1 to 100 ms. These virtual machines pro¬ 
vide real-time garbage collection, real-time fixed-priority sched¬ 
uling, priority inheritance for all Java synchronization locks, and 
priority-ordered wait and synchronization queues. The use of 
real-time virtual machines has now been proven, with a variety 
of commercially deployed products having demonstrated mil¬ 
lions of hours of successful “five nines” and higher reliability. 

Motivated by the specialized needs of developers who are 
building real-time mission-critical systems, vendors of real-time 
virtual machines have enhanced the traditional Java development 
tool chain to support capabilities that are critical to this devel¬ 
oper community. For example, traditional Java virtual machines 
support only interpreted and JIT-compiled execution of Java 
programs. But many mission-critical developers also require the 
ability to perform ahead-of-time compilation and static linking 
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of Java software components, similar to what they would do with 
more traditional C or C++ tools. 

Java developers often take the availability of high-quality 
debugger and profiling tools for granted. Many immature embed¬ 
ded virtual machine implementations do not support any com¬ 
mercial run-time performance profilers. However, experience has 
taught that understanding the performance implications associ¬ 
ated with the behavior of large, complex, dynamic systems often 
requires the ability to profile the running application. Therefore, 
mature real-time virtual machines generally provide run-time 
profiling capabilities. 

Vendors of real-time virtual machines have also found it 
necessary to support special debugging capabilities that had 
not been viewed as important to traditional desktop and enter¬ 
prise developers. For example, traditional Java virtual machine 
debuggers can only debug interpreted code. Whenever debugging 
is enabled, the virtual machine is forced to ignore its compiled 
method implementations so that the debugger can monitor the 
program behavior. Virtual machines designed for mission-criti¬ 
cal real-time systems also support the ability to debug compiled 
programs. This was motivated by the mission-critical developer 
dogma: “Test what you ship. Ship what you test!” 

It is important to have the ability to make on-the-fly adjust¬ 
ments to work buffer sizes and linked data structures, and to dy¬ 
namically load and unload Java classes. Such capabilities sup¬ 
port the dynamic reconfiguration of software that is required to 
perform the load balancing and workload (redistribution that is 
typically required in fault-tolerant and high-availability real-time 
systems. In these sorts of systems, accurate real-time garbage col¬ 
lection and deterministic thread scheduling and synchronization 
provide the solid foundations upon which the highly dynamic 
real-time system is built. 

Besides supporting the ability to balance system workloads 
and to migrate the redundant computations required to support 
warm-standby operation for fault-tolerant computation, dynamic 
class loading and unloading also empowers developers with sig¬ 
nificant new capabilities. When errors are found in deployed soft¬ 
ware, dynamic class loading supports the ability to dynamically 
load corrective patches into the running system, often without 
requiring the system to be rebooted. 

Similarly, when system functionality must be enhanced in 
response to evolving end-user requirements and/or new com¬ 
munication protocols, the new functionality can be added to the 
running system, once again without requiring any downtime. 
Finally, many mission-critical systems have the need to sup¬ 
port temporary specialized capabilities. These temporary capa¬ 
bilities can be dynamically loaded into the running system and 
then unloaded after the temporary need for this capability is no 
longer present. 

Among the various frameworks that are available to sup¬ 
port the deployment and field maintenance of Java software is 
the OSGi (Open Software Gateway Initiative) framework. This 
standard, which supports software configuration management 
on deployed systems, is available both as open-source libraries 
and commercially supported products. The OSGi framework 
keeps track of which software versions of each relevant software 
bundle are running at any given time on each deployed system. 
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OSGi management tools help automate the process of distribut¬ 
ing new software versions to targeted devices, and of automating 
the process of installation, initialization and startup of the new 
software versions. 

A typical Linux-Java deployment architecture is shown in 
Figure 1. Note that middleware is available both as native librar¬ 
ies (typically implemented in C or C++) and as Java components 
running on top of the real-time virtual machine. Almost all de¬ 
ployed soft real-time Java systems include a combination of na¬ 
tive and Java application software. In many cases, the same team 
takes responsibility for implementing both Java and native soft¬ 
ware and partitions between the two languages. This requires 
consideration of the tradeoffs in performance, memory footprint, 
real-time latency, dynamic behavior, software engineering dif¬ 
ficulty and access to low-level device hardware. In other cases, it 
is common for newly developed software that is written in Java 
to be combined with legacy software written in C or C++. Either 
approach works well. 

Linux and Java are already being used together in a va¬ 
riety of mission-critical soft real-time systems. However, 
many of these have not yet authorized public disclosure of 
their activities. 4 
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PCI Express Adapter Cards Deliver Low Latency 

Multi-processing, multi-node real-time applications need to move 
large volumes of data at high bandwidth between many systems. With 
this in mind, Dolphin Interconnect Solutions has released the D350 se¬ 
ries of PCI Express adapter cards that combine high data throughput 
with low latency. 


The single-channel x4 D351 card offers a bi-directional link speed 
of 10 Gbits/s. For greater bandwidth, the D350 and D352 cards use two bi¬ 
directional links, effectively doubling through¬ 
put to 20 Gbits/s. The low 1.4-microsecond 
application-to-application latency re¬ 
duces the overhead of inter-node 
control messages, allowing seal- 
ability for multi-node applications. 
The cards’ SCI links are hot- 
swappable connections. With the use of 
2D/3D torus configurations or switches, SCI 
technology can be used to increase failover perfor¬ 
mance and fault tolerance. The cards support both 
direct memory access (DMA) and programmed remote memory access 
(RMA). Open-source software modules include SISCI API, MPICH 
and MPICH-2. In addition, Dolphin SuperSockets software for Linux 
and Windows allows the cards to be used without the need for appli¬ 
cation modification or recompiling. Pricing for single units begins at 
$1,005 for the D352. 



Dolphin Interconnect Solutions, Sherman Oaks, CA. (818) 597-2114. 
[www.dolphinics.com]. 


Eight-Channel Encoder for Video Surveillance 
and Security Apps 

A JPEG 2000 encoder card is targeted for high reliability and 
mobile (air, bus, train, other vehicle) video applications, including 
networked/wireless video and image distribution systems and digi¬ 
tal CCTV and surveillance systems. The CTR-1471 from Parvus is a 
PC/104-Plus card supporting the J2K-ISO/IEC15444-1 image compres¬ 
sion standard. It contains a dedicated wavelet transform engine, three 
entropy codecs, an onboard memory system and embedded RISC pro¬ 
cessors to provide a complete JPEG 2000 compression/decompression 
solution. The board is able to process images at a rate of 40 Msamples/s 
in reversible mode, and at higher rates when used in irreversible mode. 



Other features include dual RISC Encoding Processors to offload 
compression overhead from system CPU and 
eight NTSC / PAL / SECAM analog 
video input channels for up to 
eight cameras. The CTR-1471 
provides programmable 
picture size, position, tilt¬ 
ing, freeze, image scaling and 
zooming. It also offers indepen¬ 
dent motion detection for each channel 
and 16 lines of digital I/O. It comes in two 
temperature ranges: 0°C to +60°C (standard) and -40°C to+85°C (ex¬ 
tended). Pricing starts at $795. 


Parvus, Salt Lake City, UT. (801) 483-1533. [www.parvus.com]. 


Conduction-Cooled 8-Port Gigabit Ethernet 
Switch PMC Card 



A new conduction-cooled, ruggedized, unmanaged, 8-port Gigabit 
Ethernet (GigE) Switch PMC card enables system designers to quickly 
and easily add high-reliability GigE switched net¬ 
works to their most demanding VMEbus and 
CompactPCI-based embedded 
applications. The PGR8 
from Curtiss-Wright Con¬ 
trols Embedded Computing 
complies with the IEEE 802.3 
Ethernet standard and provides 
integrated 8-port Layer-2 switching, delivering full wire-speed per¬ 
formance over each of its ports. The card supports up to 8K unicast 
MAC addresses. 


Networking management features on the PGR8 include a 4096-en¬ 
try ARL MAC address table with automatic learning, port trunking and 
aggregation, as well as advanced flow control and head-of-line blocking 
prevention. An optional mirroring mode enables one port to monitor 
all the other ports. Mirror mode prevents packet loss when the moni¬ 
tor port is busy or experiences very high levels of traffic, by employing 
back pressure or flow control to instruct the switch’s other ports to slow 
down. The PGR8 is designed to support rugged applications by routing 
all of the GigE ports through the PMC PN4 connector. Curtiss-Wright 
offers transition modules to provide standard RJ-45 connections for lab 
development use. Temperature range is -40° to +85°C. Single-unit pric¬ 
ing starts at $2,000. 

Curtiss-Wright Controls Embedded Computing, Kanata, Ontario. 
(613) 254-5112. [www.cwcembedded.com]. 


Flash Disk Delivers 80 Mbytes/s Sustained 
Transfer 

Designers of industrial control, telemetry and defense applica¬ 
tions are looking at 3.5-in. form-factor solid-state storage to employ 
high capacities and high sustained 
data transfer speeds. With 
that in mind, Adtron has 
introduced the latest flash 
disk in its Flashpak family, 
the A35FB 3.5-in. Serial ATA 
(SATA) disk, with sustained data 
transfer rates of up to 80 Mbytes/s. 

The company’s ArrayPro technol¬ 
ogy gives the A35FB and other flash disks 

in the Flashpak family among the industry’s highest sustained perfor¬ 
mance rates for a given capacity. Up to 128 Gbytes fit in a standard 
3.5-in. form-factor. “Clear” and “sanitize” functions ensure rapid and 
secure data elimination from the media when required. Optional de¬ 
stroy, write protection and password protection features are also avail¬ 
able. The A35FB flash disk supports either commercial (0° to 70°C) or 
industrial (-40° to +85°C) temperature ranges. Quantity pricing for an 
8 Gbyte disk is $1,900. 

Adtron, Phoenix, AZ. (602) 735-0300. [www.adtron.com]. 
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Module Adds CameraLink Connectivity to FPGA- 
Based PMC Board 

Merging data acquisition with a local, user-programmable FPGA 
gives developers a wider range of performance options when designing 
systems used for target tracking, feature recognition or real-time filter¬ 
ing. To help facilitate this, Vmetro has introduced the CAML-MOD3 
CameraLink adapter module for its high-performance Xilinx Virtex II 
Pro-based PMC-FPGA03 PMC module. 



The CAML-MOD3 uses the rapidly emerging Mini CameraLink 
(MiniCL) HDR26 connector standard. It employs the MDR26 connec¬ 
tor for traditional CameraLink equipment. In Base mode, up to two 
cameras can be attached to the CAML-MOD3/PMC-FPGA03 com¬ 
bination via separate cables. In Medium 
and Full modes, a single camera can 
be connected via two cables. The 
CAML-MOD3 supports the 
maximum CameraLink clock 
rate of 85 MHz, allowing real¬ 
time sustained data rates of up to 680 
Mbytes/s into the FPGA in Full mode. 
An efficient 64-bit PCI interface supplies the 
bandwidth necessary to transfer processed or raw 
video to the PMC host carrier. 


Developers can implement their own proprietary processing algo¬ 
rithms for capturing CameraLink data with the provided example firm¬ 
ware (VHDL) and host software (C++). Comprehensive PMC-FPGA03 
library firmware for communicating with PMC-FPGA03 board re¬ 
sources, and host library software for implementing register and DMA- 
based board communication, further simplify the development process. 
Single-unit pricing is $995. 

Vmetro, Houston, TX. (281) 584-0728. [www.vmetro.com]. 


A Low-Cost, High-Speed |2C and SPI Serial Bus 
Monitor 


A low-cost analyzer that can non-intrusively monitor high-speed 
Inter-IC (I 2 C) up to 4 MHz and Serial Peripheral Interface (SPI) up to 
24 MHz has been introduced by Total Phase. Embedded systems engi¬ 
neers can see bus transactions in real time with bit-level timing infor¬ 
mation down to 20 nanosecond resolution. The Beagle I 2 C/SPI protocol 
analyzer is appropriate both for the lab as well as on the road. Field ap¬ 
plication engineers will appreciate the Beagle analyzer’s small footprint 
for debugging in the field. Since the Beagle analyzer draws power from 
USB, there are no additional power adapters. Just plug it into a high¬ 
speed USB 2.0 port and the Beagle analyzer is ready for action. 


The royalty-free API of the 
Beagle I 2 C/SPI 
1,1 Protocol Analyzer 

is suited for auto¬ 
mated testing environments. 
Test fixtures can use the Beagle API 
to make full use of the entire feature set of the 
Beagle analyzer. The net result is a fast, efficient 
and cost-effective testing solution for manufacturing production lines. 
The Beagle I 2 C/SPI Protocol Analyzer comes with analysis software 
and royalty-free API and is priced at $300 in single unit quantities. 



Total Phase, Sunnyvale, CA (408) 850-6500. [www.totalphase.com]. 
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16-Port USB-to-Serial Servers Connect Multiple 
Peripherals 

Host computers communicating simultaneously over multiple 
ports with varied peripheral types can quickly accumulate overhead. 
A state-machine architecture can considerably reduce that overhead. 
Such an architecture has been employed in the SeaLINK+16/232 and 
SeaLINK+16/485 16-Port USB-to-serial servers from Sealevel Systems. 

Offering 16 independent 
serial ports, SeaLINK+16 
devices can be used to con¬ 
nect multiple peripherals, 
such as barcode scanners, se¬ 
rial displays and data acquisi¬ 
tion modules, to any USB port. The serial ports on each SeaLINK+16 
appear as standard COM ports to the host system, enabling compatibil¬ 
ity with legacy software. Data rates of up to 921.6 Kbits/s are supported, 
and the SeaLINK+16/485 offers RS-232/485 selectivity through cabling. 
Two USB 1.1 downstream connections are provided for daisychaining 
SeaLINK devices or interfacing standard USB peripherals. 

The SeaLINK+16/232 and SeaLINK+16/485 ship with Sealevel 
Systems’ SeaCOM suite of drivers for Windows 95/98/ME/NT/2000/ 
XP. Also included is the WinSSD application for testing and diagnos¬ 
tics. Both models are housed in rugged 1U rackmount metal enclosures. 
Standard operating temperature range is 0° to 70°C, and extended tem¬ 
perature range (-40° to +85°C) models are available. Prices start at $729 
for the SeaLINK+16/232. 

Sealevel Systems, Liberty, SC. (864) 843-4343. 
[www.sealevel.com]. 




Single-Port PCI Network Interface Card Provides 
10 Gbit Ethernet Over Copper 

Datacenters are soon expected to be able to use low-cost CAT5e/ 
CAT6 cabling for 10 GbE stacking and short-reach links (CAT5e and 
CAT6). Until now, only fiber-optic 10 GbE network adapter card (NIC) 
solutions existed. Those short-reach fiber-optic NICs cost 30% more 
than the new copper NIC from Dynatem, called the MAX Copper/10 
GbE Server Adapter (MaxCu). It 
is also expected to dramatically 
reduce installed costs based on 
materials and installation time. 

The adapter also consumes 50% 
less power than its fiber-optic 
relatives and is intended to pro¬ 
vide 10 Gbit/s performance for 
distances up to 10 meters. 

The MaxCu uses a highly integrated PCI-X Intel 82597EX 133 
MHz/64-bit 10 GbE controller coupled with a copper PHY from Vativ 
Technologies, the V10LAN transceiver/PHY. High-speed server con¬ 
nectivity once reserved for costly proprietary technologies such as high- 
performance computing clusters and grid-based computing, can now be 
achieved with the Dynatem MaxCu, which fits in a standard server PCI 
or PCI-X slot. Targeted toward the datacenter market, this product will 
let datacenter managers populate PCI slots with significantly cheaper 
alternatives without requiring replacement of existing corporate serv¬ 
ers. Single-quantity pricing starts at $1,995. 

Dynatem, Mission Viejo, CA. (949) 855-3235. [www.dynatem.com]. 
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High-Speed, Scaleable Framework Targets 
Pervasive Data Management 

A data-centric publish-subscribe (DCPS) database system operates 
across an extendable network without the access bottlenecks associated 
with a central server-based model. The SkyBoard software framework 
for managing distributed data systems from Real-Time Innovations 
(RTI) combines Oracle/TimesTen’s In-Memory Database, distributed 
database synchronization technology and RTFs Network Data Distri¬ 
bution Service (NDDS) communication middleware. 

The Oracle/TimesTen In-Memory Database lets workstations keep 
complete copies of a database’s data store units in local memory, re¬ 
ducing the need to move data to 
and from a disk during operations 
and resulting in fast data access of 
35,000 transactions per second. 
RTFs technology synchronizes da¬ 
tabase copies on multiple nodes, 
creating a distributed database and 
enabling fast access throughout a 
distributed system independent of 
the number of network nodes. RTFs 
NDDS 4.0 connects the various elements and uses a DCPS model that 
simplifies complex, high-performance data distribution while controlling 
the quality-of-service parameters needed for real-time applications. The 
infrastructure includes an industry-standard SQL interface via ODBC 
libraries, simplifying application development and combining data from 
disparate systems for real-time or post processing and analysis. 

SkyBoard includes all software elements (DDS APIs as well as the 
SQL/ODBC APIs), three developer toolkits for SkyBoard and a four- 
day product-training program. Prices start at $75,930 for RTFs NDDS 
solution with SkyBoard for Solaris, Linux and Windows (beta version). 

Real-Time Innovations, Santa Clara, CA. (408) 200-4700. 
[www.rti.com]. 
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ATCA Node Card Features XMC Interfaces 

The VITA 42 XMC standard targets the needs of high-performance 
systems in rugged environments. One of the first ATCA node cards with 
XMC interfaces is the TANC-5320 from American Portwell Technology. 
The card is supplied standard with three 64-bit/133 MHz PMC inter¬ 
faces. Two of these can be replaced with two optional XMC interfaces. 



The PICMG 3.0-compliant TANC-5320 has two Intel Xeon LV 
Nocona processors with an 800 MHz FSB and an Intel E7520 chipset. 
An Intelligent Platform Management Controller (IPMC) and dual In¬ 
telligent Platform Management Buses (IPMBs) are included for high 

reliability and system stability. 
The board has up to 16 Gbytes of 
DDR400/DDRII registered mem¬ 
ory with ECC support. For addi¬ 
tional connectivity and expansion, 
the 5320 comes with an RJ45 system 
console and a dual USB connector. 


Storage options include support 
for one 2.5” HDD at UMDA33/66/100, 
an onboard Compact Flash socket and an optional SATA 2.5” HDD 
from one of the PMC interfaces. The board supports most major operat¬ 
ing systems. Pricing starts at $1,999. 


American Portwell Technology, Newark, CA. (510) 790-9192. 
[www.portwell.com]. 


PowerPC-Based CompactPCI Single Board 
Computer Sports Latest Chipset 


A new CompatPCI-based SBC from GE Fanuc utilizes the IBM 
750GX PowerPC microprocessor offering speeds of 800 MHz or 1.0 
GHz. The CPCI-7055 uses the next-generation Mar¬ 
vell Discovery III MV64460 PowerPC chipset, 
which is an integrated system controller de- 
signed for high-performance embedded control 
applications, offering a solution for most Internet 
working applications. 

Some of the features of the CPCI-7055 single 
board computer include up to 64 Mbytes of flash 
memory, socketed or soldered down banks of flash 
are jumper-selectable for booting purposes; up to 2 
Gbytes of DDR 400 SODIMM memory with ECC 
support; and support for three 10/100/1000 Ethernet 
LANs. An IDE interface for hard disk drive sup¬ 
port allows several types of data transfers. Board 
support packages for Linux and VxWorks are 
available. The bandwidth through the system 
controller is maximized by using 2 Mbytes of 
SRAM as a revolutionary “cross-bar fabric switch” allowing “any-to- 
any” connectivity of up to 100 Gbits/s of aggregate throughput, with 
non-blocking concurrent connections among all of the peripheral chan¬ 
nels at full bus speeds. Single unit pricing starts at $3,119. 



GE Fanuc, Huntsville, AL. (800) 322-3616. 
[www.gefanuc.com/embedded]. 


Graphical Environment Supports Distributed 
Development 

The venerable and widely used LabView graphical development 
platform from National Instruments has received a major upgrade with 
the addition of distributed intelligence. Developers can now use a set of 
distributed communication and control tools along with a scalable inter¬ 
face for communicating with 
and synchronizing between 
remote intelligent devices and 
system, such as real-time pro¬ 
cessors and FPGAs. LabView 
8 offers a global system view 
of distributed development and 
open connectivity with proto¬ 
cols such as OPC, TCP/IPO 
and Modbus TCP. A system of 
shared variables abstracts the protocol transport layer to share data with 
any node in a system, including real-time nodes, historical databases 
and Web-based supervisory consoles. 

Another new feature in LabView 8 is a project-based environment 
for managing large applications and team development. LabView Project 
includes tools for multiple target management, integrated code differ¬ 
encing and source code control and multi-build management. The tool 
provides the ability for engineers and scientists to integrate LabView 
into advanced software engineering processes to comply with govern¬ 
ment process certification standards. In addition, a new Driver Finder 
tool can automatically recognize instruments and search, download and 
automatically install the appropriate driver for more than 4,000 instru¬ 
ments on NI’s driver network. Pricing starts at $995. 

National Instruments, Austin, TX. (800) 258-7022. [www.ni.com]. 
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FPGA-Based VXS Card Sports High Sample 
Rates for Demanding Sensor Apps 

A new VXS product combines FPGA technology with a dual chan¬ 
nel, 2.2 Gsample/s architecture. The Neptune 2 from TEK Microsystems 
and QuinetiQ is a 6U VME / VXS payload card for high-speed stream¬ 
ing analog-to-digital applications, combining the high sample rate with 
up to 10-bit resolution and multi-channel synchronization to single 
sample resolution. Applications such as adaptive radar beam forming, 
synthetic aperture radar, signals intelligence and electronic warfare can 
achieve new levels of accuracy. Neptune 2 combines high signal band¬ 
width with integrated onboard FPGA processing to provide improved 
performance capabilities that are especially effective in “noisier” signal 
environments where jamming or high levels of 
interference may occur. Enhanced channel-to- 
channel synchronization between multiple Nep¬ 
tune 2 cards supports applications requiring up 
to 32 channels coherent to within one sample. 
In addition, the average power consumption of 
Neptune 2 has been reduced by 20% over the 
previous version of Neptune. 

Memory and inter-processor communica¬ 
tions resources have been optimized to address 
the requirements of very high-performance, real¬ 
time digital signal processing applications. To¬ 
gether TEK Microsystems and QinetiQ provide a range of de¬ 
velopment kits, FPGA cores, software and system platforms required for 
maximum development efficiency. Single-unit pricing starts at $31,000. 

TEK Microsystems, Chelmsford, MA. (978) 244-9200. 
[www.tekmicro.com]. 


DSP Synthesis Tool Automates IP Selection 

Designers implementing DSP algorithms in FPGAs and ASICs 
have needed extensive knowledge about how specific IP blocks will 
function in a specific application. AccelChip’s new IP-Explorer tech¬ 
nology automates this process and 
has been combined with version 
2005.4 of the company’s DSP Syn¬ 
thesis software. 

The combination accommo¬ 
dates macro-architectures or func- : 

tional variants of mathematical 
building blocks such as sine, log 
and divide functions. Equipped 
with IP-Explorer, DSP Synthesis 
2005.4 automatically selects and 
inserts the optimal AccelWare DSP 
IP implementation for each function in the design, based on a variety 
of system requirements such as frequency, throughput, bit-width, area 
and sample rate. 

IP-Explorer utilizes heuristic modeling based on over 6,000 Ac- 
celChip and customer designs. AccelChip’s automated IP development 
system runs all possible combinations of AccelWare DSP IP against 
these designs, using the latest versions of the most popular design tools 
to determine silicon results. The resulting database is used by IP-Ex- 
plorer to select the optimal macro-architecture as the design’s starting 
point. During product development the design is automatically updated 
to new architectures if required by changed system requirements. Pric¬ 
ing for Version 2005.4 of AccelChip DSP Synthesis with IP-Explorer 
starts at $15,000. 

AccelChip, Milpitas, CA. (408) 943-0700. [www.accelchip.com]. 




Dual-Channel, 30 Amp DC Controller/Driver for 
Mobile Robots 


A dual-channel DC motor controller capable of directly driving up 
to 30 amps on each channel at up to 40V has been announced by Ro- 
boteq. The AX1500 is targeted at designers of mobile robotic vehicles. 
The motor controller is equally suitable to most traditional Motion Con¬ 
trol applications in machines and factory automation. Fitted on a 4.2” x 
4.2” board, the controller accepts commands from either standard R/C 
radio for simple remote controlled robot applications or serial port in¬ 
terface. Using the serial port, the AX1500 can be used to design fully 
or semi-autonomous robots by connecting it to single board computers, 
wireless modems or wireless LAN adapters. 



The two channels can be operated inde¬ 
pendently or combined to set the direction 
and rotation of a vehicle by coordinating 
the motion on each side. Using position 
sensors, the motors may be set to op¬ 
erate as heavy-duty position servos. 
The controller supports a long 
list of features, including analog 
and digital I/Os for accessories 
and sensors, thermal protection, pro¬ 
grammable acceleration, input command 
watchdog and non-volatile storage of configuration 
parameters. Single quantity pricing is $275. 


Roboteq, Phoenix, AZ. (602) 617-3931. [www.roboteq.com]. 


Generic, Off-the-Shelf Inverters for Evaluation 

When designing with LCD panels, engineers sometimes need to 
quickly evaluate a DC-to-AC inverter that powers the CCF lamps back¬ 
lighting LCDs. To fill that need, Endicott Research Group (ERG) has 
introduced the D Series Inverter, a family of generic, off-the- 
shelf inverters. 

D Series inverters can be substituted for any of the com¬ 
pany’s specific standard inverters for most LCD modules 
and can be shipped from stock. They are the same size as 
the regular part and fill 90% of pre-production design 
requirements. Once production begins, the stan¬ 
dard parts can be substituted and shipped 
under normal lead times. 

D Series inverters such as 
the 2-lamp Model DLDS60J, 
based on ERG’s LDS Series 
inverter, feature a low profile 
of less than 9 mm with display-com¬ 
patible connectors, and are designed to generate 6 mA RMS into a 350V 
to 550V load from a nominal 12V DC source. Operating temperatures 
are 0° to 70°C and onboard PWM dimming is included. Pricing in pro¬ 
duction quantities for the DLDS60J is $12.70. 

Endicott Research Group, Endicott, NY. (800) 215-5866. 
[www.ergpower.com]. 
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Boston. MA 

Los Anseles. CA 

Developing for Embedded Linux® 

Sep 20-23 

Nov 1-4 

Oct 18-21 

Jan 10-13 

VxWorks® 5.x and Tornado® 2.x 

Oct 18-21 

call for date 

Developing Embedded Linux Device Drivers 

Sep 27-30 

Oct 25-28 

Eclipse™ Plug-in Development 

Oct 11-14 

Dec 6-9 

Nov 8-11 

Jan 17-20 

VxWorks BSPs and Device Drivers 

Jan 10-13 

call for date 


For complete course descriptions go to www.mccengineering.com/traininglist.htm 
To register or find out about custom training call 940-206-5475 or email training@mccengineering.com 


VxWorks and Tornado are registered trademarks of Wind River Systems. Linux is a registered trademark of Linus Torvalds. Eclipse is a trademark of the Eclipse Foundation. 
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The Easy Way to 
Test a Board, BGAs 
and Interconnects 


Taking advantage of built-in JTAG support in most lCs is a 
straightforward way to observe the activities on the pins of BGAs 
mounted and running on boards. 


by Rick Folea, Senior Engineer 
Macraigor Systems LLC 


O ne of the most confounding tasks a hardware engineer 
comes upon happens when the first spin of a printed cir¬ 
cuit board arrives. You work for weeks on schematic cap¬ 
ture and layout. The board gets fabricated, it is populated and 
in your hands. You plug it in, turn it on, and what happens? 
Nothing. You have just entered a debug nightmare—you can’t 
get to the pins under the ball grid array (BGA), all the circuit 
traces are buried and your board is not working. Now what do 
you do? 

X-rays are an option, but they only present a static image of 
the solder joint, not a dynamic electrical report to ensure con¬ 
nectivity. It is extremely difficult to tell the difference between a 
cold solder connection and a solid one. The BGA can always be 
replaced with the hope that the problem goes away. This is usu¬ 
ally an expensive and time-consuming option and seldom yields 
results. Too bad we can’t just peel the cover of the BGA back and 
get access to the pins (Figure 1). 

Traditional boundary scan is an option, but that usually re¬ 
quires some expensive tools and the creation of test vectors and test 
executives, which can take a while, depending on the stability and 
accuracy of the design documentation. Additionally, the results are 


not dynamic—they are usually a summary of issues found with the 
board when it is in a static state, not when it is running. 

What we need is a tool that will allow us to see what all the 
pins under the BGA are doing in real time while the board is 




Get Connected with companies mentioned in this article. 

www.rtcmagazine.com/getconnected 


Figure 1 


Wish you could see what the pins under your BGA 
are doing? Now you can! 
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running; a tool that is totally transparent to the operation of the 
board. Given that, we could see if the oscillator is connected, if 
the address bus is active, if the chip enables are working, etc. It 
would be even better if that tool allowed us to manually toggle 
pins so we could drive signals out from under the BGA, in¬ 
dependent of the operation of the device under test. The good 
news is you can do this right now and you can do it quickly, 
easily and inexpensively. 



Figure 2 


The yellow scan cells (the boundary scan chain) 
allow you to capture the state of the pins and/or 
manually drive the pins independent of the internal 
logic. You can also bypass this shift register by 
sending data through the single cell BYPASS 
register instead of the long boundary scan register. 


Capturing the Pin Information 

This method takes advantage of the IEEE 1149.1 boundary 
scan capability built into most ICs. Almost every Microproces¬ 
sor, DSP, FPGA, CPLD, Ethernet switch, etc. has support for 
boundary scan via an IEEE 1149.1 JTAG interface. By accessing 
the information in that port in a non-traditional manner, you can 
see in real time what the pins on your part are doing and you 
can even manually control those pins—giving you total access to 
those otherwise inaccessible pins under the BGA! 

All integrated circuits with a JTAG port will have bound¬ 
ary scan built in. This consists of a relatively long shift register 
around the boundary of the part (the yellow boxes in Figure 2) 
and a state machine called the TAP controller to control the be¬ 
havior of the shift register (Figure 3). 

Each bit in the boundary register captures or controls some 
aspect of each pin on the device. It might capture/control the 
input buffer, the output buffer or the buffer enable. If the bit 


is a buffer enable, the register bit may monitor and/or control 
several pins. 

You simply shift commands into the TAP controller and into 
the boundary register to capture the state of every pin, and then 
shift the contents of the boundary register out the JTAG Test Data 
Out (TDO) port. If you do this repeatedly and display the results 
on the screen, you can use the data to display an activity indica¬ 
tion for every pin on every part in the JTAG chain in real time 
(Figure 4). The speed of the scan is not an issue since the display 
is analyzed visually as opposed to using test vectors. 

Here’s an example. Suppose you want to capture the contents of 
the boundary scan register and shift it out. Here are the steps starting 
in the Test-Logic-Reset state of the TAP controller in Figure 3: 

1. To capture the state of all the scan cells, transition to the 
“Capture-DR” state by clocking a “0,1,0” into the JTAG Test 
Mode Select (TMS) signal using Test Clock (TCK) for the 
clock. This takes a copy of all the scan cells associated with 
each pin function and places it in the shift register. 

2. To shift the data out, transition to the “Shift-DR” state and 
issue enough clocks to shift the entire boundary register out 
while holding TMS at “0.” 
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Figure 3 


The TAP controller consists of 16 states. The 
numbers shown on the diagram represent the 
values to place on TMS to control the transitions 
between states. Note that RESET can be reached 
from any state by holding TMS high for five TCKs. 
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Figure 4 


An example of viewing and controlling pin activity 
via boundary scan. All information shown is 
collected from the boundary scan chain and 
displayed on the screen using virtual LEDs so you 
can see what the signals are doing in real time 
while the device is running. (Screen shot courtesy 
of Macraigor Systems J-Scan product.) 



Figure 5 


An example of a full scan of a simple CPLD. Note 
the waveforms correspond to the four JTAG signals 
TCK, TMS, TDI and TDO. 


TDI is sampled on 
rising edge of TCK 



TDO appears on 
falling edge of TCK 


3. Transition the TAP controller back to a known stable state, 

usually “Run-Test-Idle. 

A similar sequence is used to shift instructions into the TAP 
controller—the only difference is you use the IR column on the 
state machine diagram instead of the DR column. 

Figure 5 is an example of a simple CPLD with a boundary 
register length of 108 scan cells. The first five clocks place the 
TAP controller into “Shift-IR” state so we can shift an instruc¬ 
tion in and tell the part what we want it to do. In this case we will 
shift in the SAMPLE instruction (a “00000001”) on Test Data In 
(TDI) with another eight clocks on TCK. The least significant bit 
is shifted in first. 

On the eighth shift, we start driving TMS to get us to the 
next state. In this example TMS is held high for three clocks 
to take us from the Shift-IR, through Exitl-IR, Update-IR and 
around to Select-DR-Scan. 

Next, TMS is held low for two more TCKs to take us through 
the Capture-DR state (where all the pin data is captured) and 
into the Shift-DR state so we can shift the captured data out of 
the device. 

The next 108 TCKs shift all the boundary scan data out of 
the device on TDO. If we were applying test vectors to pins, we 
would be shifting that data in at the same time. TDI is sampled 
on the rising edge of TCK, TDO exits on the falling edge (Figure 
6). The last two TCKs use TMS to put the device into the Update- 
DR state. From here you might go back to Run-Test-Idle or go 
do another scan. 

To apply data to pins, we would have shifted in the EXTEST 
instruction instead of the SAMPLE instruction and we would 
have shifted the data we wanted applied to the pins on the part 
on TDI. 

All of the commands are well documented in IEEE1149.1, 
and the data on boundary length and instruction register 
length of individual devices is found in the BSDL file for 
each device. You’ll find BSDL files for most devices on the 
manufacturer’s Web site. This is a text file resembling VHDL 
that completely describes every scan cell in the chain, the in¬ 
structions the TAP controller supports, the length of the scan 
chain, etc. 

The BSDL file is free, the boundary scan circuitry is already 
built into your JTAG parts, and a JTAG header probably already 
exists on your target board. You now know how to shift informa¬ 
tion into and out of the device under test so you can now write an 
application yourself to access this scan information and display 
it on the screen. 

Of course, dealing with the TAP controller is a bit tedious 
and writing an application to do all of this and display it in real 
time will take some effort and time. Fortunately, there are a 
number of boundary scan companies that offer products for 
board testing and some even offer free trials, so be sure to check 
those out. d 


Figure 6 


Close up shows that TDI is sampled on the rising 
edge of TCK and TDO appears on the falling edge 
of TCK. 


Macraigor Systems 
Brookline Village, MA. 
(617) 739-8693. 
[www.macraigor.com]. 
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T he communications industry continues to go through seeth¬ 
ing changes. Not only has the technology changed, but the 
names are changing and even the basic communications 
functions are different. Not too long ago, all phone service in the 
U.S. was provided by AT&T. Western Electric, its manufactur¬ 
ing arm, provided telephone instruments, cable and central of¬ 
fice equipment that was virtually indestructible. You picked up 
the phone and listened, and an operator asked, “number please?” 
Some areas of the country lucky enough to have advanced 
switches had “dial” phones. These activated sequences of rotary 
relays (one of which I still have buried somewhere in the attic) 
that eventually made the connection to the desired party’s phone. 
AT&T stock was the most widely held in the country and pro¬ 
vided an annuity to many pensioners in the 1940s and 1950s. 


In 2001, as we all know, things came to a screeching halt. 
Lucent stock fell to under $2 from over $70. Others in the same 
space suffered at least as badly: AT&T had a 10:1 reverse stock 
split; Nortel has been bouncing along the bottom and many oth¬ 
ers also suffered, some complete financial ruin. And some execu¬ 
tives went, or are going to jail. 

AT&T has been sold to SBC; MCI to Verizon—both off¬ 
spring of the 1984 split off of the Baby Bells. Lucent and Nortel 
are struggling, as are other suppliers to the communications in¬ 
frastructure market such as Alcatel, Siemens and others. 

A sea of change is in the offing—and it goes by the too- 
often-used term, convergence. That term adds up to providing, 
and being able to charge for, voice (wireless and wireline), data 
and all kinds of high-definition as well as other video to homes 


Communications Market: 

The Same Old Players? 


Has that ever changed! The history of AT&T itself is fasci¬ 
nating-beginning in 1876 with Elisha Gray, the one that finished 
second, getting his patent application for the telephone to the pat¬ 
ent office only hours after Alex Bell. Gray, however, left his mark 
as he went on to develop, what was at that time, the world’s larg¬ 
est electronics manufacturing company, Western Electric. AT&T 
subsequently purchased controlling interest in the company 
(1881) because demand for telephone equipment was outstripping 
the supply available from smaller manufacturers. 

After playing a critical role in World War II providing com¬ 
munications and control equipment, the company was beset by 
antitrust judgments and decrees. The resolution of an antitrust 
suit brought on by the Justice Department in 1949 and settled in 
1956, left Western Electric limited to supplying the Bell System 
and as a contract supplier to the U.S. government. Its non-tele¬ 
phone subsidiaries were sold, Westrex to Litton Industries and its 
Northern Electric subsidiary spun off as a public company now 
know as Nortel Networks 

Change struck again thanks to Judge Greene’s landmark deci¬ 
sion in 1984, which caused AT&T to split off its operating units 
(Baby Bells). Complexities in the marketplace led AT&T to totally 
reorganize its structure again in 1995 ultimately resulting in the split 
off of Lucent Technologies as a totally separate entity in 1996. 


and businesses. A battle is looming and skirmishes are already 
breaking out. The ultimate winner has probably yet to stand up 
and be counted. At the present time, Verizon and SBC are lead¬ 
ing the pack from the traditional communications companies. 
Each has a slightly different approach to achieving the goal. 
Comcast and Cox seem to be the leaders driving from the Ca¬ 
ble perspective. With a lead in customer installations providing 
video, they would appear to have the advantage—but it’s still 
very much a horserace. 

Embedded computer makers will undoubtedly find many 
opportunities from VoIP solutions over cable to providing infra¬ 
structure for fiber to the premises. ATCA is likely to be a winner 
in terms of large infrastructure requirements, while Advanced 
MC is likely to see a huge market in everything from remote 
field-based equipment to servers in large central offices (which 
won’t look anything like those of a decade ago). 

The Wall Street Journal referred to the upcoming battle of 
cable versus conventional communication providers as “The Battle 
for the Living Room.” It’s actually much bigger than that. It encom¬ 
passes how all of next-generation broadband, servers and commu¬ 
nications will be structured. Next month in this space I’ll take a 
look at this battle as it begins to surface and what technologies and 
approaches each combatant is using—they tend to vary widely, d 
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through deployment. Either way, you’ll get 
leading-edge board level solutions for the 
most demanding embedded applications. 

Tell us what you need at info@vadatech.com 
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PLUG-IN MODULES h ave never been so 
powerful, capable or easy to integrate. With 
AdvancedMCs, you simply choose the functions 
you need from our wide selection and plug them 
into your carrier. Voila: you've got a blade. And 
not just any blade, but one that can 
be easily upgraded, modified, or . flh 
serviced without disrupting the 
entire system. 

SBS has taken the lead in this dynamic 
field, as you'd expect based on our vast 
selection of PMCs. We've already created 
AdvancedMCs that cover the important telecom/ 
datacom standards and processing needs, 



and we also offer an innovative development 
chassis. Plus you can count on SBS to deliver 
AdvancedMCs that meet your future 
application requirements. 

AdvancedTCA® systems 
are only the beginning for 
AdvancedMCs. Already, these 
high-performance, very affordable 
modules have been designed into 
proprietary systems, and soon they will serve 
as the building blocks of MicroTCA™ systems. 
Modular design is changing the way systems 
are developed, and if you're a designer, that's a 
good thing. 
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